Scalable Rule-Based Representation Learning for Interpretable Classification
Authors: Zhuo Wang
Wei Zhang
Ning Liu
Jianyong Wang
Venue: NeurIPS 2021 Poster (2021)
Keywords: interpretable classification
rule-based model
representation learning
scalability

Abstract: Rule-based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a new classifier, named Rule-based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation and classification. To train the non-differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability of RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on nine small and four large data sets show that RRL outperforms the competitive interpretable approaches and can be easily adjusted to obtain a trade-off between classification accuracy and model complexity for different scenarios. Our code is available at: https://github.com/12wang3/rrl.

Scalable Rule-Based Representation Learning for Interpretable Classification
Zhuo Wang', Wei Zhang'; Ning Liu', Jianyong Wangl
'Deparmeent of Computer Science and Technology, Tsinghua University
Jiangsu Collaborative Innovation Center for Language Ability, Jiangsu Normal University Schooo of Computer Science and Technology, East China Normal University
*School of Software, Shandong University
wang-z18 @ teaallstsinguua.du.... {zhangwei. thu2011 1, victorliues) @ gmail.com, jianyong @ wwwtinnguaeedu...o
Abstract
Rule-based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a new classifier, named Rule-based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation and classification. To train the non-differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on nine small and four large data sets show that RRL outperforms the competitive interpretable approaches and can be easily adjusted to obtain trade-off between classification accuracy and model complexity for different scenarios. Our code is available at: https: // /////gtttt gighuu....... com/ 2omm//2aang//rx
Introduction
Although Deep Neural Networks (DNNs) have achieved impressive results in various machine learning tasks (Goodfellow et al.) 2016). rule-based models, benefiting from their transparent inner structures and good model expressivity, still play an important role in domains demanding high model interpretability, such medicine, finance, and politics (Doshi-Velee Velez and Kim 2017). In practice, rule-based models can easily provide explanations for users to earn their trust and help protect their rights (Molnar 2019 Lipton| 2016). By analyzing the learned rules, practitioners can understand the decision mechanism of models and use their knowledge to improve or debug the models (Chu et al.) 2018). Moreover, even if post-hoc methods can provide interpretations for DNNs, the interpretations from rule-based models are more faithful and specific (Murdoch et al. 2019). However, conventional rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures, which limit their application scope. To take advantage of rule-based models in more scenarios, we urgently need to answer such question: how to improve the scalability of rule-based models while keeping their interpretabilit??
"Coreeponning authors.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Studies in recent years provide some solutions to improve conventional rule-based models in different aspects. Ensemble methods and soft/fuzzy rules are proposed improve the performance and
scalability of rule-based models but at the cost of model interpretability (Ke et al. 2017 [Beimmn
2001; Irsoy et al. 2012). Bayesian frameworks are also leveraged to more reasonably restrict and
adjust the structures of rule-based models (Letham et al. [015: Wang et al. 2017: Yang et al. 2017). However, due to the con-differentiable model structure, they have to use methods like MCMC or
Simulated Annealing. which could be time-consuming for large models. Another way to improve rule-based models is to let nigh-performance but complex model (e.g., DNN) teach a rule-based model (Frosst and Hinton 2017 Ribeiro et al. 2016). However, learning from complex model requires soft rules, or the fidelity of the student model is not guaranteed. The recent study Wang et al.) (2020) tries to extract hierarchical rule sets from tailored neural network. When the network is large, the extracted rules could hehave quite differently from the neural network and become useless in most cases. Nevertheless, combined with binarized networks (Courbariaux al. 2015), it inspires us that we can search for the discrete solutions of interpretable rule-based models in continuous space leveraging effective optimization methods like gradient descent.
In this paper, we propose a novel rule-based model named Rule-based Representation Learner (RRL) (see Figure lap. We summarize the key contributions as follows:
To achieve good model transparency and expressivity, RRL is formulated as a hierarchical model, with layers supporting automatic feature discretization, rule-based representation learning in flexible conjunctive and disjunctive normal forms, and rule importance evaluation.
To facilitate training effectiveness, RRL. exploits a novel gradient-based based discrete model train- ing method, Gradient Grafting, that directly optimizes the discrete model and uses the gradient information at both continuous and discrete parametric points to accommodate more scenarios.
To ensure data scalability, RRL utilizes improved logical activation functions to handle high- dimensional features. By further combining the improved logical activation functions with tailored feature binarization layer, realizes the continuous feature discretization end-to-end.
We conduct experiments on nine small data sets and four large data sets to validate the advantages, i... good accuracy and interpretability. of our model over other representative classification models. The benefits of the model's key components are also verified by the experiments.
2 Related Work
Rule-based Models. Decision tree, rule list, and rule set are the widely used structures in rule-based models. For their discrete parameters and non- non-differentiable structures, we have to train them by employing various heuristic methods (Quinlan 1993; Breiman 2017: Cohen 1995; Wei et al. 2019), which may not find the globally best solution or solution with close performance. A Alternatively, train 2020), which could take too much time on large data sets. In recent studies, Bayesian frameworks are leveraged to restrict and adjust model structure more reasonably (Letham et al.) 2015 Wang et al.
them with search algorithms (Wang et al.) 2017 Angelino et al.) 2017: Lin et al.
2017 Yang et al.) 2017). Lakkaraju et al. (016)) learns independent if-then rules with smooth local search. However, except for heuristic methods, most existing rule-based models need frequent itemsets mining and/or long-time searching, which limits their applications. Moreover, it is hard for these rule-based models to get comparable performance with complex models like Random Forest.
Ensemble models like Random Forest (Breiman 2001 and Gradient Boosted Decision Trees (Chen and Guestrin| 2016: Ke et al.) 2017) have better performance than the single rule-based model. However, since the decision is made by hundreds of models, ensemble models are commonly not considered as interpretable models (Hara and Hayashi 2016). Soft or fuzzy rules are also used to improve model performance (Irso et al. 2012 Ishibuchi and Yamamoto 2005), but non-discrete rules are much harder to understand than discrete ones. Deep Neural Decision Tree YYngg ct al. 2018) is tree model realized by neural networks with the help of soft binning function and Kronecker product. However, due to the use of Kronecker product, it is not scalable with respect to the number of features. Other studies try to teach the rule-based model by complex model, e.g., DNN, or extract rule-based models from complex models (Frosst and Hinton 2017; Ribeiro et al., 2016 Wang et al.) 2020). However, the fidelity of the student model or extracted model is not guaranteed.
Gradient-based Discrete Model Training. The gradient- discrete model training methods mainly proposed to train binary quantized neural networks for network compression and
2

---

Overall Structure. A Rule-based Representation Learner (RRL), denoted by F. is hierarchical model consisting of three different types of layers. Each layer in RRL not only contains a specific number of nodes but also has trainable edges connected with its previous layer. Let (4(1) denote the 1-th layer of RRL, u, indicate the j-th node in the layer, and represent the corresponding number of nodes, I E (O..... in the layer. For ease of expression, we denote this vector by (!), There are only one binarization layer, i.e., 2)(0), and one linear layer, i.e., 24(2), in RRL,, but the number of middle layers, i.e., logical layers, can be flexibly adjusted according to the specific situation. The logical layers mainly aim to Discrete Logical Layer Example:
Forward
Logical Liyer
Linear Layer
Loss(.)
Logical Layer
Confunction Layer
Disunction Layer
Correspondngg Rules:
Logical Layer
Discrete Model
Continuous
up
Conjunction Layer
Digunction Layer
Model
10
10 1
10 Vuy
Sinarization
Continuuus input C2 Binery Input B,
(b)
Figure 1: (a) A Rule-based Representation Learner example. The dashed box shows an example of discrete logical layer and its corresponding rules. (b) A simplified computation graph of Gradient Grafting. Arrows with solid lines represent forward pass while arrows with dashed lines represent backpropagation. The green arrow denotes the grafted gradient, copy of the gradient represented by the red arrow. After grafting, there exists backward path from the loss function to the parameter 0.
acceleration. Courbariaux et (2015 2016 propose to use the Straight-Through Estimator (STE) for binary neural network training. However, STE requires gradient information at discrete points, which limits its applications. ProxQuant (Bai al. 2018) formulates quantized network training as regularized learning problem and optimizes it via the prox-gradient method. ProxQuant can use gradients at non-discrete points but cannot directly optimize for the discrete model. The RB method (Wang et al., 2020) trains neural network with random binarization for its weights to ensure the discrete and the continuous model behave similarly. However, when the model is large, the differences between the discrete and the continuous model are inevitable. Gunbel-Softmax estimator (Jang et al., 2016) generates a categorical distribution with differentiable sample. However, it can hardly deal with a large number of variables, e.g., the weights of binary networks, for it is biased estimator. Our method, i..., Gradient Grafting, different from all the aforementioned works in using gradient information of both discrete and continuous models in each backpropagation.
3 Rule-based Representation Learner
Notation Description. Let = {(X1, Yi). (XN, Yx)} denote the training data set with N instances, where X; is the observed feature vector of the i-th instance with the j-th entry Xt,, and Y, is the corresponding one-hot class label vector, i {I,...,NN N.. Each feature value can be either discrete continuous. All the classes take discrete values, and the number of class labels is denoted by M. We use one-hot encoding to represent all discrete features as binary features. Let C, E I and B, {0, 1).h denote the continuous feature vector and the binary feature vector of the i-th instance respectively. Therefore, X; = C. @ B, where @ represents the operator that concatenates two vectors. Throughout this paper, Boolean variable. Thus each dimension of a binary feature vector corresponds to a Boolean variable.
(True) and (False) represent the two states of
, L}.
The output of the I-th layer is a vector containing the values of all the nodes
learn the non-linear part of the data, while the linear layer aims to learn the linear part. One example of RRL is shown in Figure
When we input the i-th instance to RRL, the binarization layer will first binarize the continuous feature vector C₁ into new binary vector C, Then, C₁ and B, are concatenated together as (0) and inputted to the first logical layer. The logical layers designed to automatically learn data
3

---

representations using logical rules, and the stacked logical layers can learn rules in more complex forms. After going through all the logical layers, the output the last logical layer can be considered as the new feature vector to represent the instance, wherein each feature corresponds to one rule formulated by the original features. As such, the whole RRL composed of feature learner and a linear classifier (linear layer). Moreover, the skip connections in RRL can skip unnecessary logical layers. In what follows, the details of these components will be elaborated.
3.1 Logical Layer
Considering the binarization layer needs the help of its following logical layer to binarize features in an end-to-end way, we introduce logical layers first. As mentioned above, logical layers can learn data representations using logical rules automatically. To achieve this, logical layers are designed to have a discrete version and continuous version. The discrete version is used for training, testing and interpretation while the continuous version is only used for training. It is worth noting that the discrete RRL indicates the parameter weights of logical layers take discrete values (i.e., 0 or 1) while the parameter weights and biases of the linear layer still take continuous values.
Discrete Version. One logical layer consists of one conjunction layer and one disjunction layer. In discrete version, let R(() and (()) denote the conjunction and disjunction layer of 2((1) (1 e (1,2...., L- 13) respectively. We denote the i-th node in R(0) by (!) and the i-th node in S!!) by s!") Specifically speaking, node r:!)) corresponds the conjunction of nodes in the previous layer connected with " while node s; corresponds to the disjunction of nodes in previous layer
connected with o! Formally, the two types of nodes are defined as follows:
(1-1)
W
where W11.0) denote the adjacency matrix of the conjunction layer R(O) and the previous layer(e(1-1) and w:(1.)
w:ll.) f.j {(0,1, 1}. W((,0) indicates there exists an edge connecting r(!) to u. uf-"), otherwise 1.j Similarly, W(I1)) is the adjacency matrix of the disjunction layer (( and (((-1),
Similar to neural networks, we regard these adjacency matrices as the weight matrices of logical layers. u = r()) s()), where r!) and s are the outputs of R( and (! respectively.
The function of the logical layer is similar to the notion "level" in Wang et al. 2020). However, one level in that work, which actually consists of two layers, can only represent rules in Disjunctive Normal Form (DNF), while two logical layers in RRL can represent rules in DNF and Conjunctive Normal Form (CNF) at the same time. Connecting nodes in R( with nodes in S(1-1), we get rules in CNF, while connecting nodes in (!)) with nodes in R we get rules in DNF. The flexibility of logical layer is quite important. For instance, the length of CNF rule (a) V (2) A.. A (a2n-1 1 V a2n) is 2n, but the length of its corresponding DNF rule (a, A a3 ... A a2n- 1 V... V (az A a...... azn) n 2n, which means layers that only represent DNF can hardly learn this CNF rule.
Continuous Version. Although the discrete logical layers have good interpretability, they are hard to train for their discrete parameters and oindifferrnniable structures. Inspired by the training process of binary neural networks that searches the discrete solution in continuous space. we extend the discrete logical layer to a continuous version. The continuous version is differentiable. and when we discretize the parameters of a continuous logical layer, we can obtain its corresponding discrete logical layer. Therefore, the continuous logical layer and its corresponding discrete logical layer can also be considered as sharing the same parameters, but the discrete logical layer needs to discretize the parameters first.
Let (7 denote the continuous logical layer, and R()) and S(() denote the continuous conjunction and disjunction layer respectively, e (1,2...., 1}. Let (p.(1.0 and (fr(1.1) denote the weight
matrices of (2)) and $(1) respectively. W/((.") vil"l") e [0. 1]. To make the whole Equation differentiable, we leverage the logical activation functions proposed by Payani and Fekri (019);
Conj(h, W.)
Fo Wa.)), j=1
II
Disj(h, W.) 1 II (1 - Fa (h,,,,,)),
(2)

---

where Fo w) = w(1 h) and Fa(h, ()) = h w. In Equation vectors, then Conj(h, W;) = Aw.,=1 hj and (isj(h, W;) Vw.,=I h, Fo w) and Fa w) decide how much h, would affect the operation according to W... If W, = 0, h, would have no effect the operation. Equation cann be considered extensions t-norm fuzzy logic (Hájek)
2 if h and W; are both binary
2013). After using continuous weights and logical activation functions, the nodes in R00 and S()) are defined as follows:
(3) Now the whole logical layer is differentiable and can be trained by gradient descent. However, the above logical activation functions suffer from the serious vanishing gradient problem. The main reason can be found by analyzing the partial derivative of each node w.r.t. its directly connected weights and w.r.t. its directly connected nodes as follows:
p Conj(úl-1), (1
$ $(") Disj(úul-1) W.!l.))
de(!)
OF!
(ū) (in--1)-1 1)- II Ang
Folug
(1-1) Wi.
II F (1-1) w:ll")) (4)
ow-(. 1.j
Oug (1-1)
3.k Since ug 11 and W.(L0) are in the range [0, 1], the values of F-(-) in Equation are in the range [0, 1]
as well. If n 1 is large and most of the values of P.(-) are not 1, then the derivative is close to 0
due to the multiplications (See Appendix D for the analysis of s. ( Wang et al. (2020) tries to use weight initialization to make Fo(-) close to T at the beginning. However, when dealing with hundreds of features, the vanishing gradient problem still inevitable.
Improved Logical Activation Functions. We found that using the multiplications to simulate the logical operations in Equation Dis the main reason for vanishing gradients and propose an improved design of logical activation functions. One straightforward idea is to convert multiplications into additions using logarithm, e.g., log [TT" Fc Wi,))) = Σÿ= (F.FF.,,, ))) However, after taking the logarithm, the logical activation functions in Equation 2 cannot keep the characteristics of logical operations any more, and the ranges of Conj(-) and Disj(-) are not [o, 1]. To deal with this problem, we need a projection function fix it. Apparently, the inverse function of log(æ), i.e., e".
is not suitable.
For the projection function g. three conditions must be satisfied: (i) g(0) e (ii) -- g(x) lim ex 0. (iii) lim g(ø) O. Condition (i) and (ii) aim to keep the range and tendency of logical activation functions. Condition (iii) aims to lower the speed of approaching zero when xo. In this work, we choose g(x) = -1+. as the projection function, and the improvement
of logical activation functions can be summarized as the function P(v) -1+1g((0) The improved conjunction function Conj+ and disjunction function Disj+ are given by:
a
P Chn W.) = I (Falh,, Wo)) Disj (h, W.) = 1- P II (1 Pa(h, +es) (5) j=1
where is small constant, e.g., 10-10 10_ The improved logical activation functions avoid the vanishing gradient problem in most scenarios and are much more scalable than the originals. Moreover, considering that dP(v) (v) when in Equation 5 is extremely large, dF(v) may be de
very close to 0 due to (v). One trick deal with it replacing P(e) with (() for P can lower
the speed of approaching O while keeping the value range and tendency.
3.2 Binarization Layer
The binarization layer is mainly used to divide the continuous feature values into several bins. By combining one binarization layer and one logical layer. we can automatically choose the appropriate bins for feature discretization (binarization), i.e., binarizing features in an end-to-end way.
For the j-th continuous feature be binarized, there are lower bounds (T,,1-..,, Tx and upper bounds (H........ H, All these bounds are randomly selected (e.g., from uniform distribution) in
the value range of the j-th continuous feature, and these bounds are not trainable. When inputting
one continuous feature vector c, the binarization layer will check if c, satisfies the bounds and get the following binary vector:
Q = ig(c, Tra)...,, .9(6, - TTx), -z), ..)...., ",),,.,,((HH.. -c;],,
(6)
5

---

where g(z) = 1xs0. If we imput the i-th instance, i.e., c C₁ then C, = Q1 + Q2 Qm and (0)) = C; + B, After imputting u to the logical layer 4 the edge connections between ((2)) and () indicate the choice of bounds (bins). For example, if r("") is connected to the nodes corresponding to a(e; - -T,1) and g(H3,2 - C2), then ((1) contains the bin (7 < &c,) (cg < Hzz2). If we replace (()) with s; in the example, we can get (T₁, < <C)) (c, < H1 should be noted that, in practice, if T, 2 H2,2, then r, O. and if T,11 < H2,2. then s 1. When using the continuous version, the weights of logical layers are trainable, which means we can choose bounds in an end-to-end way. For the number of bounds is 2/ times of features, which could be large, only logical layers with improved logical activation functions are capable of choosing the bounds.
3.3 Gradient Grafting
Although RRL can be differentiable with the continuous logical layers, it is challenging to search for a discrete solution in a continuous space (Qin et al. 2020p. One commonly used method tackle this problem is the Straight- Through Estimator (STE) (Courbariaux et al.) 2016). The STE method needs gradients at discrete points to update the parameters. However, the gradients of RRL at discrete points have no useful information in most cases (See Appendix E Therefore STE is not suitable for RRL. Other methods like Prox (Bai et al. 2018) and Random Binarization (Wang et al., 2020) cannot directly optimize for the discrete model and be scalable at the same time.
Inspired by plant grafting, we propose a new training method. called Gradient Grafting, that can effectively train RRL. In stem grafting, one plant is selected for its roots, i.c. rootstock, and the other plant is selected for its stems, i.e., scion. By grafting, we obtain a new plant with the advantages of both two plants. In Gradient Grafting, the gradient of the loss function w.r.t. the output of discrete model is the scion, and the gradient of the output of continuous model w.r.t. the parameters of continuous model is the rootstock. Specifically, let 0 denote the parameter vector and Of denote the parameter vector at step L. ((x) = 1.5005 is the binarization function that binarizes each dimension of x with 0.5 as the threshold. Let y and Y denote the output of the continuous model f and discrete model F respectively, then Y = F(0', X), Y = F(q(0"), X)). The parameters update with Gradient Grafting is formulated by:
g 8¹ AL(Y) 0r aY
Og:
where 17 is the learning rate and L(-) is the loss function. One simplified computation graph of Gradient Grafting is shown in Figure [b for intuitive understanding.
Gradient Grafting can directly optimize the loss of discrete models and use the gradient information at both continuous and discrete points, which overcomes the problems occurring in RRL training when using other gradient- discrete model training methods. The convergence of Gradient Grafting is verified in the experiments (See Figure 3).
3.4 Model Interpretation
After training with Gradient Grafting, the discrete RRL be used for testing and interpretation. RRL is easy to interpret, for we can simply consider as a feature learner and a linear classifier. The binarization layer and logical layers describe the new features. The linear classifier, i.e., the linear layer, makes decisions based on the new features. We can first find the important new features by the weights of the linear layer, then understand each new feature by analyzing its corresponding rule. One ad of RRL is that it can be easily adjusted by the practitioners obtain trade-off between the classification accuracy and model complexity. Therefore, RRL can satisfy the requirements from different tasks and scenarios. There are several ways limit the model complexity of RRL. First, we can reduce the number of logical layers in RRL, i.e., the depth of RRL. and the number of nodes in each logical layer, i.e., the width of RRL. Second, the L./L.2 regularization can be used during training to search for an RRL with shorter rules. The coefficient of the regularization term in the loss function can be considered as a hyperparameter to restrict the model complexity. After training, the dead nodes detection and redundant rules climination proposed by Wang et al. (2020) can also be used for better interpretability.
the feature learner, and they use logical rules to build and

---

Table 1: 5-fold cross validated F1 score (%) of comparing models on all 13 datasets. represents that RRL significantly outperforms all the compared interpretable models (t-test with p < 0.01). The first seven models interpretable models, while the last five are complex models.
Dataset
RRI.
C4.5
CART
SBRI.
CORFI
CRS
LR
SVM
PLNN(ML.P)
LightGiMM
XGiooost
adult
80.72
77.77
77.06
79.85
70.56
80.95
78.43
63.63
73.55
79.22
80.36
80.64
bank-markoting
76.32'
71.24
71.38
72.67
66.86
73.34
69.81
66.78
72.40
72.67
75.28
74.71
banknote
100.00"
97.85
94.44
100.00
chess
78.83
79.90
79.15
26. 26.44
24.86
80.21
33.06
79.58
77.85
75.00
80.58
80.66
connect-4
71.23'
61.66
61.24
48.54
51.72
65, 65,88
49.87
69.85
64:55
62.72
70.53
70.65
keRRoog
64.32
92.34
96.51
magic04
86.33"
82.44
81.20
82.52
77.37
50.87
75.72
79.43
83.07
$6.48
86.67
86.69
tic-tec-tee
100.00
91,70
94,21
98.39
98.92
99.77
98.12
98.07
98. 26
98. 98.37
99.89
99 99.89
wine
98.23
95.48
94.39
95.84
97.43
97.78
76,07
98.31
98.44
activity
98.17
94,24
93.35
11.34
51.61
5.05
98. 47
98.67
98.27
97.80
99.41
99.38
dota2
60.12'
52,08
51.91
34.83
46.21
56.31
59.34
57.76
59.46
57.39
58.81
58.53
facebook
80.76
81.50
31.16
34.93
11.38
62
89.43
87.49
85.87
88.90
fashion
89.01*
80.49
79.61
47,38
38.06
66.92
84,55
84.46
89.36
88.35
89.91
89.82
ArgRank
2.77
8,23
8.92
9.31
7.92
6.77
5.77
2.77
4 Experiments
In this section, we conduct experiments to evaluate the proposed model and answer the following questions: (i) How is the classification performance RRL.? (ii) How is the model complexity of RRL? (iii) How is the convergence of Gradient Grafting compared to other gradient-based discrete model training methods? (iv) How is the scalability of the improved logical activation functions?
4.1 Dataset Description and Experimental Settings
Dataset Description. We took nine small and four large public datasets to conduct our experiments, all of which are often used to test classification performance and model interpretability (Dua and Graff
2017 Xiao et al. 2017 Anguita et al., 2013 Rozemberczki et al. 2019). Appendix [ summarizes the statistics of these 13 datasets. Together they show the data diversity, ranging from 178 to 102944
instances, from 2 to 26 classes, and from to 4714 14 original features. See Appendix A for licenses.
Performance Measurement. We adopt the F1 score (Macro) as the classification performance metric since some of the data sets are imbalanced, i.e., the numbers of different classes are quite different. We udopt 5-fold gross-validation to evaluate the classification performance more fairly. The average rank of each model is also adopted for comparisons of classification performance over all the data sets (Demšar 2006). Considering that reused structures exist in rule-based models, e.g., one branch in Decision Tree can correspond several rules, we use the total number of edges instead of the total length of all rules as the metric of model complexity for rule-based models. See Appendix C for details about the experiment environment and parameter settings of all models.
4.2 Classification Performance
We compare the classification F1 score (Macro) of RRL with six interpretable models and five complex
models. C4.5 (Quinlan 1993). CART (Breiman 2017). Scalable Bayesian Rule Lists (SBRL) (Yang)
al. 2017). Certifiably Optimal Rule Lists (CORELS) (Angelino et al., (017), and Concept Rule
Sets (CS) (Wang et al.,
2020) are rule-based models. Logistic Regression (LR) (Kleinbaum et al.,
2002) is a linear model. These six models are considered interpretable models. Piecewise Linear
Neural Network (PLNN) (Chu et al.] 2018). Support Vector Machines (SVM) (Scholkopf and Smola
200I), Random Forest (Breiman] 2001). LightGBM (Ke et al. 2017). and XGBoost (Chen and Guestrin 2016) are considered complex models. PLNN is Multilayer Perceptron (MLP) that adopts piecewise linear activation functions, c.g., ReLU (Narr and Hinton 2010). RF, LightGBM, and
XGBoost are ensemble models. See Appendix C for the parameters tuning.
The results are shown in T and the first nine data sets are small data sets while the last four are large data sets. We can observe that RRL performs well on almost all the data sets and gets the best results on data sets. The two-tailed Student's t-test (p<0.01) is used for significance testing, and we can observe that RRL significantly outperforms all the compared interpretable models on 8 out of 13 data sets. The average rank of RRL is also the top three among all the models. Only two complex models that use hundreds of estimators, i.c., XGBoost and LightGBM, have comparable results with
7

---

bank-marketing
dota2
facebook
60
90-
75.0
80-
F7 Score
72.5
Score
70.0
CARTIOES1 CARTISSE31 8E31
50
CARTICaLI
CARTTD 03)
67.5
CART6-0331
50
CARTTS 600)
65.0
45
CARTIL2211
40
CARTIS 6013
CAS
CRS
30
CRS
60.0
Woboortte-
221
Nillocetio 1000
Wo 100
57.5
35
4
10- 6
12
8
12
6
10
12
log(#edges)
log(# (og(#))ges)
log(#edges)
Figure 2: Scatter plot of F1 score against log(#edges) for RRL and baselines on three datasets (see Appendix K for other datasets).
RRL. For Table I the Friedman statistic is 73.84, corresponding to a p-value of 2.26 x 10-11, Hence the null hypothesis of significant differences is rejected. We then calculate pairwise comparisons using Conover post-hoc test, and RRL significantly outperforms all the compared interpretable models with p<0.01. Comparing RRL with LR and other rule-based models, we can see RRL can fit both linear and non- non-linear data well. CRS performs well on small data sets but fails on large datasets due to the limitation of its logical activation functions and training method. Good results on both small and large data sets verify RRL has good scalability. Moreover, SBRL and CRS do not perform well on continuous feature data sets like letRecog and magic04 for they need preprocessing to discretize continuous features, which may bring bias to the data sets. On the contrary, RRL overcomes this problem by discretizing features end-to-end.
4.3 Model Complexity
Interpretable models seek keep low model complexity while ensuring high accuracy. To show the relationships between accuracy and model complexity of RRL and baselines, we draw scatter plots of F1 score against log(Hedges) for rule-based models in Figure (see Appendix [oo other data sets). The baselines are typical models in different categories of methods with good trade-offs between accuracy and model complexity. For RRL, the legend markers and error bars indicate means and standard deviations, respectively, of F1 score and log(#edges) across cross-validation folds. For baseline models, each point represents an evaluation of one model, on one fold, with one parameter setting. Therefore, in Figure 2 the closer its corresponding points are to the upper left corner, the better one model is. To obtain RRL. with different model complexities, we tune the depth and width of RRL and the coefficient of L2 regularization term. The value in CART(-), c.g., (ART() denotes the complexity parameter used for Minimal Cost-Complexity Pruning (Breiman 2017). and a higher value corresponds to simpler tree. We also show results of XGBoost with 10 and 100 estimators.
In Figure 2 on both small and large data sets, we can observe that if we connect the results of RRL, it will become a boundary that separating the upper left corner from other models. In other words, if RRL has close model complexity with one baseline, then the Fl score of RRL will be higher. If the F1 score of RRI. is close to one baseline, then the model complexity of RRI. will be lower. It indicates that RRL can make better use of rules than rule-based models using heuristic and ensemble methods in most cases. The results in Figure alsoo RRL by setting the model structure and the coefficient of L2 regularization term. In this way, the practitioners are able to select an RRL with suitable classification performance and model complexity for different scenarios, which is crucial for practical applications of interpretable models.
also verify that we can adjust the model complexity of
4.4 Ablation Study
Training Method for Discrete Model. To show the effectiveness of Gradient Grafting for training RRL, we compare it with other representative gradient- discrete model training methods, i.e., STE (Courbariaux et al., 2015 (2166), ProxQuant (Bai et al., 2018) and RB (Wang et al.) 2020), by training RRL with the same structure. Hyperparameters are set to be the same for each method except exclusive hyperparameters, e.g., random binarization rate for RB, are fine-tuned. The training loss of the compared discrete model training methods and Gradient Grafting are shown in Figure and we can see that the convergence of Gradient Grafting faster and stabler than other methods on all data sets. As mentioned in Section 3.3 RRL has little useful gradient information at discrete points,
8

---

connect-4
activity
facebook
0.5
Grack
Gra
loss (log scale)
Dran
loss (log scale)
PraxCuant
GracCratingg
loss (lag scale)
N
ProeOvent
FB
Grad
-0.5
-1.0
-1.5
-2.0
-2.
30
50
10
2
iter. (1e4)
iter. (1e4)
iter. (1e4)
Figure 3: Training loss of three compared discrete model training methods and Gradient Grafting with or without improved logical activation functions three data sets.
2-1e-07
1-1o-0
Weight
Rule
0
-1225 < balance 2006.1 manital= married campaign SA poutcome success previous 0
weight
weight gh1
C758
17572 balanos 7018.7 A mariial in married contact - teleponoxx 6 day < 27 provious xt
A-0, 0.0000
2=0.001
age 38 balance < 7028.7 marital married A campaign SA pdays 104 poutcome sucoee
36 < age balance 1 campaign 7 day pdays previous 0
age 20 -0991 < balaance 58137 campaign SAA pdays
previous
weight
weight
(b)
(a)
Figure 4: (a) The distribution of weights in the linear layer of RRLs trained on the bank-markeling data set with the same model structure but different X, where À is the coefficient of the L.2 regularization term. (b) Logical rules obtained from RRL trained on the bank-marketing data set.
thus RRL trained by STE cannot converge. Due to the difference between discrete and continuous RRL, RRL trained by ProxQuant and RB cannot converge well as well.
Improved Logical Activation Functions. We also compare RRL trained by Gradient Grafting with without improved logical activation functions. The results are also shown in Figure 3 and GradGrafting(NN) represents RRL using original logical activation functions instead of improved logical activation functions. We can observe that the original activation functions work well on small data sets but fail on the large data set activity while the improved activation functions work well on all data sets, which means the improved logical activation functions make RRL more scalable. It should be noted that GradGrafting(NI) works well on the large data set facebook. The reason is facebook is very sparse data set, and the number of in each binary feature vector is less than 30 (See Appendix D for detailed analyses).
4.5
Case Study
We show how the learned RRL looks like by case studies. Take the bank-marketing data set as an example. We first show the distribution of weights in the linear layer of the trained RRLs in Figure 4a Each weight in the linear layer corresponds to one rule. For a better viewing experience, we show the normalized absolute values of weights. The model structures of these RRLs are the same, but different coefficients of the L2 regularization term, denoted by X, lead to different model complexities. We can observe that, when À is small, which means the RRL is more complex, there are many rules with small weights. These small weight rules are mainly used to slightly adjust the outputs. Hence, they make the RRL more accurate but less interpretable. However, in practice, we can ignore these small weight rules and only focus on rules with large weights first. After analyzing rules with large weights and having a better understanding of the learned RRL and the data set, we can then understand those less important rules gradually. When À is large, the number of rules is small, and we can directly understand the whole RRL rather than understanding RRL step by step.
In Figure 4b] we show the learned rules, with high weights, of one RRL trained on the bank-marketing data set (see Appendix for the fashion data set). These rules are used to predict if the client will subscribe a term deposit by telesales. Different types of features are marked in different colors, e.g., purple for previous behaviours of the bank. We can clearly see that middle-aged married persons with low balance are more likely to subscribe a deposit, and the previous behaviour of the bank would also affect the client. Then the bank can change its strategies according to these rules.

---

Conclusion and Future Work
We propose new scalable classifier, named Rule-based Representation Learner (RRL), that can automatically learn interpretable rules for data representation and classification. For the particularity of RRL, we propose new gradient-basee
discrete model training method, i.e., Gradient Grafting. that directly optimizes the discrete model. We also propose improved design of logical activation functions to increase the scalability of RRL and make RRL capable of discretizing the continuous features end-to-end. Our experimental results show that RRL enjoys both high classification perfor- mance and low model complexity on data sets with different scales. limited structured data, when dealing with unstructured data using RRI, we need to convert them into structured data first. For example, RRL needs to convert a 2D image into a ID vector, which could lose spatial information. We will extend RRL to suit more unstructured data future work.
the current design of RRL
6 Acknowledgments
This work was supported in part by National Key Research and Development Program of China under Grant No. 2020YFA0804503, National Natural Science Foundation of China under Grant No. 61521002 and 62072182, and Beijing Academy of Artificial Intelligence (BAAI).
References
Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, and Cynthia Rudin. 2017. Learning certifiably optimal rule lists for categorical data. The Journal of Machine Learning Research 18, (2017), 8753-8833
Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz, 2013. A public domain dataset for human activity recognition using smartphones.. Esann.
Yu Bai, Yu-Xiang Wang, and Edo Liberty. 2018. Proxquant: Quantized neural networks via proximal operators. arXiv preprint arXiv:1810.0056 (2018).
Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5-32.
Leo Breiman. 2017. Classification and regression trees. Routledge.
Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd aem sigkdd international conference on knowledge discovery and data mining. 785-794.
Lingyang Chu, Xia Hu, Juhua Hu, Lanjun Wang, and Jian Pei. 2018. Exact and consistent in- terpretation for piecewise linear neural networks: A closed form solution. In SIGKDD, ACM, 1244-1253.
William W Cohen. 1995. Fast effective rule induction. In MLP. Elsevier, 115-123.
Matthieu Courbariaux, Yoshua Bengio, and lean-Pierre David. 2015. Binaryconnect: Training deep neural networks with binary weights during propagations. In NeurIPS. 3123-3131.
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran EI- Yaniv, and Yoshua Bengio. 2016. Bina- rized neural networks: Training deep neural networks with weights and activations constrained tot or-1. arXiv preprint vrXiv:1602. 02830 (2016).
Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning research 7, Jan (2006), 1-30.
Finale Doshi-Veler and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint
(2017).
Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. http::/archive ics uci edu/mm
Nicholas Frosst and Geoffrey Hinton. 2017. Distilling neural network into a soft decision tree. arXiv preprint ar 11110:1711.09784 (2017).
10

---

Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. 2016. Deep learning. Vol. 1. MIT Press.
Petr Hájek. 2013. Metamathematics of fuzzy logic. Vol. 4. Springer Science & Business Media.
Satoshi Hara and Kohei Hayashi. 2016. Making tree ensembles interpretable. arXiv preprint arXiv:1606.05390 (2016).
Ozan Irsoy, Olcay Taner Yıldız, and Ethem Alpaydın. 2012. Soft decision trees, In Proceedings of the 2 International Conference on Pattern Recognition (ICPR2012). IEEE, 1819-1822.
Hisao Ishibuchi and Takashi Yamamoto. 2005. Rule weight specification in fuzzy rule-based classification systems. IEEE transactions on fuzzy systems 13, (2005), 428-435.
Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv: 1611.01144 (2016).
Guolin Ke, Qi Meng. Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie- Yan Liu. 2017. Lightgbm: highly efficient gradient boosting decision tree. In Advances Neural Information Processing Systems. 1156-3154.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. 2002. Logistic regression. Springer.
Himabindu Lakkaraju, Stephen H Bach, and Jure Leskovec. 2016. Interpretable decision sets: A joint framework for description and prediction. In SIGKDD. ACM, 1675-1684.
Benjamin Letham, Cynthia Rudin, Tyler H McCormick, David Madigan, et al. 2015. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics 9, 3 (2015), 1350-1371.
Jimmy Lin, Chudi Zhong, Diane Hu, Cynthia Rudin, and Margo Seltzer. 2020. Generalized and scalable optimal sparse decision trees. In International Conference Machine Learning. PMLR, 6150-6160.
Zachary C Lipton. 2016. The mythos of model interpretability. arXiv preprint 100.000600003490 (2016).
Christoph Molnar. 2019. Interpretable Machine Learning. https://christophhe wwt.iii....t........... 1o/ interpretable-nl-ee
W James Murdoch, Chandan Singh, Karl Kumbier. Reza Abbasi-Asl, and Bin Yu. 2019. Interpretable machine learning: definitions, methods, and applications. arXiv preprint 1901::9001.0599 (2019).
Vinod Nair and Geoffrey Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML. 807-814.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in neural information processing systems. 8026-8037.
Ali Payani and Faramarz Fekri. 2019. Learning algorithms via neural logic networks. arXiv preprint arXiv: (2019).
Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, and Nicu Sebe. 2020. Binary neural networks: A survey. Pattern Recognition (2020), 107281.
Ross Quinlan. 1993, C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

---

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. SIGKDD. ACM, 1135-1144.
Benedek Rozemberczki, Carl Allen, and Rik Sarkar. 2019. Multi-scale Attributed Node Embedding. arXiv:1909. 13021 [es.LG]
Bernhard Scholkopf and Alexander J Smola. 2001. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press.
Tong Wang, Cynthia Rudin, Finale Doshi Velez, Yimin Liu, Erica Klampfi, and Perry MacNeille. 2017. A bayesian framework for learning rule sets for interpretable classification. JMLR 18, (2017), 2357-2393.
Zhuo Wang, Wei Zhang, Ning Liu, and Jianyong Wang. 2020. Transparent Classification with Multilayer Logical Perceptrons and Random Binarization. In AAAI. 6331-6339.
Dennis Wei, Sanjeeb Dash, Tian Gao, and Oktay Gunluk. 2019, Generalized linear rule models. In International Conference on Machine Learning. PMLR, 6687-6696.
Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-MNIST.: Novel Image Dataset for Benchmarking Machine Learning Algorithms. LtXiv:cs...G/110.0077 [cs.LG]
Hongyu Yang, Cynthia Rudin, and Margo Seltzer. 2017. Scalable Bayesian rule lists. In ICML. JMLR. org, 3921-3930.
Yongxin Yang, Irene Garcia Morillo, and Timothy M Hospedales. 2018. Deep neural decision trees. arXiv preprint 1XXX::88060099800 (2018).
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you deseribe the limitations of your work? [Yes] See Section 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes] There is no potential negative societal impact of our work.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL.)? [Yes] See Appendix
A for the code, data, and instructions.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section and Appendix [
(c) Did you report error bars (e-g., with respect the random seed after running experi-
ments multiple times)? [Yes] See Figure and Appendix K
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix [
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
12

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] The data we are using do not contain any personally identifiable information or offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "XytOkTTwaM8", "reviewer": "Reviewer_v1eM", "summary": "The paper introduces a gradient-descent and network-based training method to learn a rule-based interpretable classifier by training concurrently a continuous model and a discretized version.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "A discrete model and a continuous one are trained concurrently, but the way the gradient information by the continuous model used to affect (support to train) the discrete model is not presented clearly.\n\nUsing k random upper bounds and k random lower bounds for each continuous attribute in binarization step maybe is not a good practice, since the bounds are selected randomly and continuous attributes can be different in ranges and number of distinct values. The hyper-parameter k is added to the set of inherent hyper-parameters of the network-based training methods that makes the algorithm more difficult to apply.\n\nThe RRL algorithm is stated to be scalable; however how its efficiency (running time) and consumed memory are that is not reported. The following features can make the algorithm less scalable in term of running time.\n- To support processing directly continuous attributes, the way of binarization used by the RRL increases very much the dimension of input vectors (at Binarization layer).\n- To support both CNF and DNF, RRL algorithm maintains concurrently Conjuction part and Disjunction part in each Logical layer that also makes Logical layer bigger.\n- A discrete model and a continuous one are trained concurrently. \n\nHow is the behavior of training loss of Gradient Grafting on the other 11 datasets, it could be nice if its training loss for the other 11 dataset is supplemented in Figure 3.\n\nA model with high classification performance in F1-score does not imply it will be high in Accuracy metric. Choosing Accuracy or F1-score metric to evaluate classification performance of a model depends on that (TP, TN) or (FP, FN) is more important in a particular application rather than on whether the class distribution is balance or not. It could be more interesting to report the performance in accuracy metric to see how the ranking will be.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have not addressed this question, nor would I see how they could have done that.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "PHbL3-kq93q", "reviewer": "Reviewer_Uttc", "summary": "The authors propose a new classifier - called Rule-based Representation Learner - that learns interpretable non-fuzzy rules. The model consists of 3 main modules: \n- the binarization layer\n- the logical layer(s)\n- the linear layer. \nFurther, the authors propose a new training algorithm called Gradient Grafting.\nThe model was tested tested on 13 datasets and compared with six interpretable models and five complex models.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Novelty**: novel. The authors propose a new method that automatically learns non-fuzzy rules for data representation and classification. \n\n**Clarity**: clear. The paper is easy to follow. \n\n**Quality**: the paper seems sound, however I have the following concerns: \n\n- the authors propose Gradient Grafting, and their experiments show that it converges. However, the authors do not provide any intuition on why this new training method should work better than the others.\n\n- The authors, while implicitly using them, never mention the t-norms. According to [1], a fuzzy logic can be defined upon a certain t-norm (triangular norm) representing an extension of the Boolean conjunction. A t-norm is a function $T : [0,1]^2 \\to [0,1]$ that, for every $a,b,c \\in [0,1]$, satisfies the following properties: \n$$\nT(a,b) = T(b,a),  \\qquad\nT(a,T(b,c)) = T(T(a,b),c), \n$$\n$$\na \\le b \\to T (a, c) \\le T (b, c)  \\qquad\nT(a,1) = a, \\qquad\nT(a,0) = 0.\n$$\nSo, if we consider the product t-norm we can represent:\n$$\nx \\wedge y, \\qquad x \\cdot y, \n$$\n$$\n\\neg x, \\qquad 1-x. \n$$\n$$\nx \\vee y = \\neg (\\neg x \\wedge \\neg y), \\qquad x+y -xy = 1 - (1-x)(1-y)\n$$\nWhich are exactly the functions used in Eq. (2) in the paper. The authors should add a discussion on the relationship between Eq. (2) and t-norms in order to justify why they model conjunction and disjunction in the proposed way. Further, they should discuss whether the improved logical activation functions retain the properties of the t-norms. \n\n- Why do the authors need to add the small constant $\\epsilon$ in Eq. 5?\n\n- the authors mention the following \"trick\": '' One trick to deal with it is replacing $\\frac{\\partial \\mathbb{P}(v)}{\\partial v}$ with $\\frac{P(P^2(v))}{v}$ for $P$ can lower the speed of approaching 0 while keeping the value range and tendency.'' Has this been tested? \n\n- the authors compare the models by average rank, however, as proposed by Demsar himself in [2], when comparing multiple models models over multiple datasets the Friedman test + post-hoc test should be used to verify the statistical significance of the results. \n\n- Parameter settings not specified. It would be highly beneficial to have a table stating exactly the number of layers and the number of neurons per layer for each dataset (and hence the final number of rules obtained). The interpretability of the model itself highly depends on this. Further, it would also help the reader to include in Table 4 in the appendix not only the average length of the rules, but also the minimum and maximum length of the rules.  \n\n- I am not sure I understand the graphs in Fig. 2. Regarding RRL: the authors plot how the F1-score varies according to the number of edges. Regarding the other models: what does the $x$ axis represent for the other models? For example, consider the bank-marketing dataset why is CART's F1-score plotted on left of the graph, while XGBoost F1-score gets plotted on the right?\n\n[1] Petr Hajek. Metamathematics of fuzzy logic, volume 4. Springer Science & Business Media, 2013.\n\n[2] Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "IwN7wEU9sC_", "reviewer": "Reviewer_nbhN", "summary": "This paper presents a new scalable classifier, named Rule-based Representation Learner (RRL), that can automatically learn interpretable rules for data representation and classification.  A new gradient-based discrete model training method, i.e., Gradient Grafting, is proposed as well that directly optimizes the discrete model. RRL's performance on structured data seems promising wrt experiments performed.\n\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Novelty:\n1. Although influenced by Wang et al [2020], the proposed gradient grafting technique based RRL is novel as it can handle vanishing gradient and large no of features.\n\nStrengths:\n1. Code available at https://github.com/anonymous-rrl/rrl\n2. The layers are well explained.\n3. The proposed gradient grafting technique seems to work quite well in the specified experiments.\n4. Case study proves application in practical scenario.\n\nWeakness:\n1. Explanation on 'interpretability' part is missing while accuracy is established. Using rules increases expectation interpretation.\n2. Experiments are sound, however the worst accuracy obtained should have been reported as well across datasets along with corner cases of failures.\n3. Limitations of the paper (unstructured data) are only mentioned as one liner in Section 5. This needs expansion as a subsection including implementation level and formulation level limitations and assumptions.\n\nRelevance to NeurIPS:\nThis paper is relevant to Neural Information Processing System conference, as we are moving slowly towards Neuro-symbolic AI papers.\n\nCorrectness:\nClaimed formulations and results seem to be correct.\n\nClarity:\nPaper is well written.\n\nRelated work Comparison:\nComparison is fine wrt state-of-the-art.\n\nEffort: The amount of work done in writing paper, coding, experimenting, supplementary material is appreciable.\n\nSuggestions:\n1. It is better to do a Train-Eval-Test split to avoid biased results following guidelines in Andrew Ng's \"Machine Learning Yearning\" book.\n2. Try to bring in the mathematical guarantees, upper-lower bound of the testing stage.\n3. Apart from Figure 1 inner blocks, a pipeline of the whole process would have made the picture clear in a single figure.\n\nVerdict: This paper falls in the above acceptance threshold area and there are still some concerns in terms of analysis of results in terms of interpretability. Differentiation with related work was expected to be more rigorous. However, this is a good work. Papers on rule based learning are rare wrt the general bulk of usual deep learning papers - so it is recommended to accept the paper to maintain flavour of NeurIPS, provided other reviewers agree.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is no issue wrt negative Societal Impact, hence skipped.", "review_text": "", "extra_scores": {"needs_ethics_review": "Yes", "ethics_review_area": ["Research Integrity Issues (e.g., plagiarism)"], "time_spent_reviewing": "3"}, "extra_text": {"ethical_concerns": "This paper can be found in https://openreview.net/forum?id=UwOMufsTqCy as an ICLR 2021 submission that was rejected. If authors are same, then no issue, else plagiarism as per Docoloc is 42%. See:- https://www.docoloc.de/d88c4d5162dd4b21eeb6add446e2b5bbCEuFHG85rulAsphkx9/en/konto.hhtml?dogetresult=49. Please crosscheck this if its fine for a green signal. I assume it's fine."}}, {"review_id": "1x4JQfa2FRk", "reviewer": "Reviewer_WXKm", "summary": "This submission proposes a new rule-based learner for classification. The learner uses discrete logical learners to obtain the rules. In order to obtain differentiability, the authors also propose a continuous approximation to the discrete layers. After a discussion about avoiding vanishing gradients problem, the authors conduct numerical experiments on a set of datasets. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors propose a new network learner using discrete and continuous layers. Although their main objective is to come up with an interpretable machine learning approach, they also obtain a method that performs remarkably well according to the numerical experiments. I believe the strength of the proposed method is not its interpretability but its accuracy as it beats (or performs quite close to) several sophisticated algorithms including SVM, XGBoost and Random Forest.  \n\nWhen it comes to the interpretability-accuracy trade-off, I have several questions and comments:\n\n1) The number of obtained rules with the proposed approach is still quite large. Do you obtain less number of rules than the ones obtained with the interpretable models like CRS and CORELS?\n\n2) Do the authors use the resulting set of rules (with the weights for classification) after training, or do they use the trained model with the continuous weights for classification? I guess this is related to whether they use the discrete or the continuous weights in the logical layers during testing.\n\n3) How does the random bound selection in the binarization layer affect the interpretation? Given different bounds for two trained models on the same dataset, could the interpretations be completely different?\n\n4) In Figure 2, the performance of CART is not good for bank-marketing dataset. But its performance looks quite good in Table 1. What is the difference?\n\n5) The authors report the weights with absolute values. But clearly some of the weights could be negative. How can one interpret negative weights? \n\n6) In Section 4.5, the authors state that \"However, in practice, we can ignore these weight rules and only focus on rules with large weights first.\" However, in this case the accuracy of the resulting set of rules does not necessarily match the accuracy obtained with all the rules.  Therefore, this statement is not exactly true.\n\n7) The submission claims to  propose a scalable rule-based learner but the related work considered in rule-based methods do not involve some recent work after 2017. For instance \nWei et al., (2019), Generalized linear rule models.\nLin et al., (2020) Generalized and scalable optimal sparse decision trees\n\nOverall, the numerical experiments show that the authors have come up with a quite accurate algorithm. However, as I listed above, the arguments supporting the interpretability of the algorithm are not convincing.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes they have.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a new scalable classifier, named Rule-based Representation Learner (RRL), that can automatically learn interpretable rules for data representation and classification. A new gradient-based discrete model training method, i.e., Gradient Grafting, is proposed as well that directly optimizes the discrete model. RRL's performance on structured data seems promising wrt experiments performed.

Overall, the reviewers agree that there is merit in the proposed approach and they all support the acceptance of the manuscript. I also looked at aspects of the manuscript and found the experimental evaluation very well organized and thorough. Moreover, I particularly like the results properly showing the trade-offs of complexity and performance and the comparison with other methods according to these two measures. This can convincingly, and in a quantitative manner show, that the proposed method can produce short explanations (i.e., small number of rules/edges) while still performing relatively well. Moreover, when allowing complex explanations, the model performs provides state-of-the-art results when compared to other approaches that produce less interpretable predictors. I believe that approaches like the proposed ones will be important to get AI accepted with application fields, such as medicine or finance.

In summary, there are strong arguments supporting the acceptance of the manuscript and I recommend accepting the manuscript.
Decision: Accept (Poster)