From: ICLR.cc/2021/Conference/Program_Chairs
title: Final Decision
decision: Reject
comment: This paper falls in the borderline area and there are still some concerns (for instance by AnonReviewer5 and AnonReviewer2) that deserve further treatment. Given that most ideas can only be validated in experiments (as the results are not theoretical), some points that remain are the comparison with other approaches (there are reasonable comparisons, but there are very famous contenders missing such as xgboost, ok that LightGBM is, but why not the other?), details about the tuning, the significance of results (practical and statistical is not complete/detailed enough), and the reasoning about situations with many rules and interpretability seems to be worth exploring/discussing further.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: General Response: Revision Updated 
comment: We thank all reviewers for their valuable suggestions with effort and time. We revised our manuscript according to reviewers' comments. Here we briefly summarize the major update in the revision, and please refer to the responses of each reviewer for more details.

We have revised our manuscript to include the following changes:

- We have changed Section 3.1 to clarify the relationship between the continuous logical layer and the discrete logical layer.
- We have replaced the average F1 score with the average rank of all the baselines in Table 1.
- We have added a new table, i.e., Table 3, containing the training time of RRL on all 13 datasets to further verify the scalability of our model.
- We have added a new table, i.e., Table 5, to show the average length of rules in RRL trained on different datasets.
- We have added a new table, i.e., Table 4, to show the distribution of weights in the linear layer of RRL trained on the bank-marketing data set.
- We have added and discussed some references to related works according to the reviews.

Please don't hesitate to let us know of any additional comments on the manuscript or the changes. Thank you again.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: Response to AnonReviewer2 (Part 1/2)
comment: We appreciate the reviewer's effort. Here are our responses to the comments.

>Q1: Are the results given for test set? If so, what is the train-test percentage?

Yes, the results are given for test set. We use 5-fold cross-validation (mentioned in Section 4.1), therefore the train-test percentages are 80\% and 20\%.

---

>Q2: What are the computation times? Without these figures, it is hard to assess the scalability of the proposed method.

The computation times of RRL is similar to neural networks for their computations are very similar. The training time of RRL on all the datasets (400 epoch on small datasets and 100 epoch on large datasets) is:

|dataset | adult | bank-marketing | banknote | chess | connect-4 | letRecog |  magic04 | tic-tac-toe | wine | activity |   dota  | facebook | fashion  | 
|:-----:|:--------------:|:--------:|:-----:|:---------:|:--------:|:--------:|:-----------:|:----:|:--------:|:-------:|:--------:|----------|----------|
| RRL   | 1h22m55s       | 1h0m49s  | 7m45s | 29m40s    | 2h20m41s | 2h16m24s | 3h22m37s    | 1m3s | 16s      | 1h2m24s | 1h58m42s | 2h27m23s | 7h32m52s |

---

>Q3: The numbers of edges that you report range from 50 to 1000. These values seem quite large. How does this affect the interpretability?

Indeed a large number of edges could affect the interpretability of RRL. However, to obtain an acceptable accuracy, a certain number of rules are essential especially when the datasets are quite complex. Figure 2 and 6 show that among models with a comparable accuracy with black-box models like random forest, RRL is the most interpretable model in most cases, which indicates RRL is the best choice in most cases for the scenarios demanding good interpretability, accuracy and scalability. It also indicates that RRL can make better use of rules than models using heuristic and ensemble methods. Moreover, RRL can obtain a trade-off between accuracy and interpretability easily. When the number of rules is large, practitioners can interpret the more important rules first according to the weights in the linear layer, which is much easier than interpreting the whole model. The average length of rules in RRL trained on different datasets is as follow:

|       dataset       | adult | bank-marketing | banknote | chess | connect-4 | letRecog | magic04 | tic-tac-toe | wine | activity | dota2 | facebook | fashion |
|:-------------------:|:-----:|:--------------:|:--------:|:-----:|:---------:|:--------:|:-------:|:-----------:|:----:|:--------:|:-----:|----------|---------|
| avg rule len | 5.71  | 5.62           | 3.56     | 7.03  | 12.44     | 5.91     | 5.05    | 3.07        | 2.11 | 6.67     | 4.37  | 38.28    | 34.53   |

We can see that except the facebook and fashion datasets, the average length of rules is less than 13 (most are less than 7), which means understanding one rule is easy and understanding the rules one by one in the order of weights is feasible. The average length of rules of RRL trained on the facebook and fashion datasets is large for facebook and fashion are actually two unstructured datasets, e.g., the fashion dataset is an image classification dataset. How to interpret RRL trained on the fashion dataset is shown in Appendix D.
In addition, the L1/L2 regularization can also be used during training to restrict model complexity. Practitioners can adjust the width and depth of RRL to get a trade-off between accuracy and interpretability.

---

>Q4: In Figure 4, only five clear rules are reported. What is the distribution of the weights of the remaining rules?

The distribution of the weights of all the rules in the RRL shown in Figure 4 is as follow:

| abs(weight) | [0, 0.1) | [0.1, 0.2) | [0.2, 0.3) | [0.3, 0.4) | [0.4, 0.5) | [0.5, 0.6) | [0.6, 0.7) | [0.7, 0.8) | [0.8, 0.9) | [0.9, 1.0) |
|:-----------:|:--------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|
| probability | 0.0781   | 0.1875     | 0.3125     | 0.2422     | 0.0859     | 0.0469     | 0.0078     | 0.0313     | 0.0000     | 0.0078     |

---

>Q5: How do you decide various design parameters; such as, number of layers (n\_l), number of bins for feature discretization (k), number of layers (L)?

We have used validation sets to tune hyperparameters of RRL as mentioned in Section 4.1. Grid search is also used for tuning. The number of bins $k \in \\{5, 10, 15, 20\\}$. The number of nodes in logical layers $n_l\in \\{16, 32, 64, 128, 256, 512, 1024\\}$. The number of layers (L) depends on the number of logical layers, and we use no more than three logical layers.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: Response to AnonReviewer2 (Part 2/2)
comment: >Q6: As you need a binarization layer to divide the continuous features into bins, can't we just say that the method works only with discrete features? This is how it would be presented by other rule-learning methods.

RRL can deal with continuous features for it realizes the continuous feature discretization in an end-to-end manner by combining the binarization layer with a logical layer. The binarization layer provides all the possible bounds and the logical layer choose the appropriate bounds to build the rules. Therefore, the binarization layer and logical layer are both important for the feature discretization, and that the difference between RRL and other rule-learning methods that can only deal with discrete features.

---

>Q7: I guess Section 3.2 is superficial as it is straightforward to split a continuous feature into bins. Am I missing something here?

The goal of Section 3.2 is not only to introduce the binarization layer, but also to emphasize the combination of the binarization layer and the logical layer. Indeed the binarization layer is simple, but it is effective for feature discretization with the help of one logical layer. As we mentioned in Section 4.2, the experimental result in Table 1 indicates that the way RRL used to discretize continuous feature is better than the features discretization preprocessing used by CRS and SBRL for the preprocessing may bring bias to the data sets.

---

>Q8: How do your results compare against the results obtained with other rule/tree learning methods based on (integer) linear optimization? Some names from that field are Bertsimas, Rudin, Gunluk.

Could you please provide some paper here for these three researchers have too many works.

---

>Q9: Can you guarantee that the resulting set of rules covers the entire feature space? In other words, is it possible that a test sample is not classified with the output set of rules?

If one test sample is not covered by all the rules, then the value of the last logical layer will be all zero, which means the linear layer will use the bias to classify this test sample. Therefore, there is no need to worry that one sample may not be covered by all the rules.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: Response to AnonReviewer4
comment: We appreciate the reviewer's effort. Here are our responses to the comments.

>Q1: However, it is questionable whether the difference in accuracy between decision trees such as C4.5 and RRL in the experimental results is crucial. If the difference in accuracy is as small as this, the ease of interpretation of the decision tree may be superior to that of the RRL. Alternatively, the choice of experimental data may not be appropriate to demonstrate the superior performance of RRL.

We performed a two-tailed Student’s t-test(p<0.001) for significance testing and found that our method RRL significantly outperforms decision trees on 10 out of 13 data sets in Table 1. Therefore, it is unfair to say the difference between the accuracy of RRL and decision trees is small. What's more, according to [1], we can use functionally-grounded evaluation method, e.g., total number of edges, to compare the interpretability of RRL and decision trees for they are all rule-based models and both have reused structures. Experimental results shown in Figure 2 and 6 indicate that the interpretability of RRL is close to decision trees when their accuracies are all acceptable. As we mentioned in Section 4.1, the datasets we used are often used to test classification performance and model interpretability [2][3][4][5] and the result of t-test also indicates that these datasets are appropriate to demonstrate the superior performance of RRL.

[1] Doshi-Velez, Finale, and Been Kim. "Towards a rigorous science of interpretable machine learning." arXiv preprint arXiv:1702.08608 (2017).
[2] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml.
[3] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
[4] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz. A public domain dataset for human activity recognition using smartphones. In Esann, 2013.
[5] S´ergio Moro, Paulo Rita, and Bernardo Vala. Predicting social media performance metrics and evaluation of the impact on brand building: A data mining approach. Journal of Business Research, 69(9):3341–3351, 2016.

---

>Q2: I also didn't understand why the fuzzy/soft rule sacrifices the model interpretability. Could it be that the fuzzy representation is intended to be closer to human subjectivity, and thus aid in intuitive understanding?

When we say the fuzzy/soft rule sacrifices the model interpretability, we indicate that the interpretability of discrete logical rules is better than fuzzy/soft rules. One important metric of interpretability is simulatability, which means a person can easily simulate the model or one part of the model [6]. However, the operation or computation of fuzzy/soft rules is more complex than the discrete rules. For example, one node in the soft decision tree is a logistic regression [7] while one node in the normal decision tree is just a split by feature values. The fuzzy sets used by fuzzy rules make it harder to simulate too. [8] also indicates that fuzzy knowledge representation must confront the problem of preserving the overall system accuracy, thus often yielding a trade-off between accuracy and interpretability. In practice, the fuzzy set close to human subjectivity is hard to obtain, especially for the datasets with only class labels information available.

[6] Lipton, Zachary C. "The mythos of model interpretability." Queue 16.3 (2018): 31-57.
[7] Irsoy, Ozan, Olcay Taner Yıldız, and Ethem Alpaydın. "Soft decision trees." Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012). IEEE, 2012.
[8] Alonso, Jose M., Ciro Castiello, and Corrado Mencar. "Interpretability of fuzzy systems: Current research trends and prospects." Springer handbook of computational intelligence. Springer, Berlin, Heidelberg, 2015. 219-237.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: Response to AnonReviewer1
comment: Many thanks for your strong review, we’re glad to hear you are pleased with the paper! Thanks for arguing in our favor! We believe that our paper will inspire other research on interpretable machine learning.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: Response to AnonReviewer5 (Part 1/3)
comment: We appreciate the reviewer's effort. Here are our responses to the comments.

>Weakness 1: There is limited novelty in the proposed model. The continuous/discrete conjunction and disjunction operations used in the logical layers are the same as those used in Wang et al. (2020), however they are arranged differently in this method.

The continuous/discrete conjunction and disjunction operations used in Wang et al. (2020) is proposed by [1]. As we mentioned in the continuous version of the logical layer in Section 3.1 and Appendix A, conjunction and disjunction operations proposed by [1] suffer from the serious vanishing gradient problem, and Wang et al. (2020) cannot tackle this problem when dealing with hundreds of features. We propose a new logical activation function using additions instead of multiplications to simulate the logical operations and avoid the vanishing gradient problem. The experimental results shown in Section 4.4 also verify the effectiveness of our logical activation function.

[1] Ali Payani and Faramarz Fekri. Learning algorithms via neural logic networks. arXiv preprint arXiv:1904.01554, 2019.

---

>Weakness 2: a) The connection between the continuous version of the model and the discrete version is weak. b) One of the claimed contributions is that the discrete model is optimised directly, but this does not seem to be the case in practice---the continuous version of the model is optimised. c) It is also unclear how the continuous-valued weights are binarized in order to work with the discrete version of the model.

a) In fact, the connection between the continuous version of the model and the discrete version is quite strong for they share the same parameters. As shown in Figure 1b in our paper, the parameters of the discrete model and continuous model are both $\theta$. The difference is the discrete model needs to binarize $\theta$ first, i.e., using $q(\theta)$ to do logical operations. Here $q(\mathbf{x})= \mathbf{1}_{\mathbf{x}>0.5}$ is the binarization function that binarizes each dimension of $\mathbf{x}$ with 0.5 as the threshold.

b) The Equation 7 in Section 3.3 shows how Gradient Grafting directly optimizes the discrete model. The Equation 7 is $\theta^{t+1}=\theta^{t}-\eta \frac{\partial L(\bar{\mathbf{y}})}{\partial \bar{\mathbf{y}}}\cdot \frac{\partial \hat{\mathbf{y}}}{\partial \theta^{t}}$, where $\bar{\mathbf{y}}=\mathcal{F}(q(\theta^{t}),X)$ is the output of the discrete model and $\hat{\mathbf{y}}=\hat{\mathcal{F}}(\theta^{t},X)$ is the output of the continuous model. We can see that, by Gradient Grafting, what we optimized is the loss of the discrete model, i.e., $L(\bar{\mathbf{y}})$, and that is why we claim the discrete model is optimised directly. The backprop path is built by both the discrete model and the continuous model because its part from discrete model, i.e., $\frac{\partial L(\bar{\mathbf{y}})}{\partial \bar{\mathbf{y}}}$, describes how the loss of discrete model changes more precisely and its part from continuous model, i.e., $\frac{\partial \hat{\mathbf{y}}}{\partial \theta^{t}}$, describes how the parameters affect the middle states more precisely.

c) After training by Gradient Grafting, we can directly binarize the parameters $\theta$ with 0.5 as the threshold to obtain the parameters of discrete model, i.e., $q(\theta)$.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: Response to AnonReviewer5 (Part 2/3)
comment: >Weakness 3: The results in Figure 2 indicate that typically a large number of rules are learned, yielding models that are unlikely to be interpreted reliably. The log scale on the x axis of the plots is somewhat deceptive---I find it hard to believe that models with between 64--256 terms are interpretable. I think further investigating what tradeoff exists between accuracy and interpretability is necessary before the paper can be accepted.

First, we need to clarify that the log scale on the x axis of the plots in Figure 2 does not mean to be deceptive. The log scale is aimed to obtain a better viewing experience and let readers know the order of magnitude more intuitively. Second, it is hard to get a specific bound that models with the number of rules more than this bound are not interpretable. And for some complex datasets, a certain number of rules are essential to get an acceptable accuracy. Therefore, a better way to check whether a rule-based model is interpretable is by comparison, and that is why we draw a 2D plot of accuracy and complexity (Figure 2 and 6). Through this 2D plot, we can easily compare the RRL with the decision trees, CRS and random forests. We can see that among models with a comparable accuracy with black-box models like random forest, RRL is the most interpretable model in most cases. RRL is designed for the scenarios demanding good interpretability, accuracy and scalability, and Figure 2 and 6 indicate that RRL is the best choice for this kind of scenarios in most cases. Third, RRL can obtain a trade-off between accuracy and interpretability easily. When the number of rules is large, practitioners can interpret the more important rules first according to the weights in the linear layer, which is much easier than interpreting the whole model. The average length of rules in RRL trained on different datasets is as follow:

|       dataset       | adult | bank-marketing | banknote | chess | connect-4 | letRecog | magic04 | tic-tac-toe | wine | activity | dota2 | facebook | fashion |
|:-------------------:|:-----:|:--------------:|:--------:|:-----:|:---------:|:--------:|:-------:|:-----------:|:----:|:--------:|:-----:|----------|---------|
| avg rule len | 5.71  | 5.62           | 3.56     | 7.03  | 12.44     | 5.91     | 5.05    | 3.07        | 2.11 | 6.67     | 4.37  | 38.28    | 34.53   |

We can see that except the facebook and fashion datasets, the average length of rules is less than 13 (most are less than 7), which means understanding one rule is easy and understanding the rules one by one in the order of weights is feasible. The average length of rules of RRL trained on the facebook and fashion datasets is large for facebook and fashion are actually two unstructured datasets, e.g., the fashion dataset is an image classification dataset. How to interpret RRL trained on the fashion dataset is shown in Appendix D. In addition, the L1/L2 regularization can also be used during training to restrict model complexity. Practitioners can adjust the width and depth of RRL to get a trade-off between accuracy and interpretability too.

---

>Comment 1: Comparing average accuracy is generally not a good idea. It is more informative to compare the average rank of each method across datasets.

Thank you for the advice and we will replace the average F1 Score in Table 1 with the average rank. The average rank of each method across datasets is:

|         |  RRL | C4.5 | CART | SBRL |  CRS |  LR  |  SVM | PLNN(MLP) | GBDTe=100 | RFe=10 | RFe=100 |
|:-------:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:---------:|:---------:|:------:|:-------:|
| AvgRank | 2.15 | 7.62 | 8.38 | 8.54 | 5.85 | 7.23 | 7.15 | 4.69      | 2.92      | 6.54   | 4.54    |
|         |      |      |      |      |      |      |      |           |           |        |         |
---

>Comment 2: Can the authors elaborate on how hyperparameters for each of the competing methods are chosen?

We have used validation sets to tune hyperparameters of all the baselines as mentioned in Section 4.1. We used sklearn to implement LR, and carefully tune the penalty, tol, C and solver. For decision tree, we tune its max depth, min samples split and min samples leaf. For SBRL, we tune its lambda, rule\_minlen, rule\_maxlen, eta, iters, nchain, minsupport\_pos and minsupport\_neg. For CRS, we tune its random binarization rate, learning rate, learning rate decay rate, batch size, model structure (depth and width) and weight decay. For PLNN, we tune its learning rate, learning rate decay rate, batch size, network structure, weight decay. For SVM, we tune C, gamma and tol. For RF, we tune min\_samples\_split and min\_samples\_leaf. We used LightGBM to implement GBDT, and tune the learning\_rate and num\_leaves. Grid search is also used for tuning. Hence, the baseline results should be reliable.

From: ICLR.cc/2021/Conference/Paper1045/Authors
title: Response to AnonReviewer5 (Part 3/3)
comment: >Comment 3: The computational graph for gradient grafting seems related (yet distinct) to the Gumbel softmax estimator---perhaps the authors could discuss this relationship further?

Gumbel softmax is not suitable for RRL because it is used to generate a categorical distribution while the weight of the continuous RRL is not a probability. As we mentioned in Section 3.1, one weight of the continuous RRL is used to tell how much the node it attached would affect the logical operation, and its corresponding weights in the discrete RRL is deterministic (hard binarization with a threshold). Due to the particularity of logic operation, the change of one discrete weight may cause a big change of the whole model. Therefore, if we use Gumbel softmax instead, it is similar to use a soft binarization and it is hard to evaluate the loss of the discrete RRL. For the big difference between the soft binarized RRL and the hard binarized RRL, the RRL using Gumbel softmax can hardly converge during the training. Moreover, Gumbel softmax is mainly used for discrete outputs rather than discrete weights, and we usually do not use Gumbel softmax to train the binary neural networks.

---

>Comment 4: The authors could do more to discuss related work on training decision trees and rules using gradient-based optimisation. E.g., XGBoost, DNDT, and SGT.

Thank you for your advice. XGBoost is one implementation of GBDT, we have already discussed GBDT in our paper and used another implement of GBDT, i.e., LightGBM [2], in our experiments. DNDT is a tree model realised by neural networks with the help of soft binning function and Kronecker product. However, due to the use of Kronecker product, it is not scalable with respect to the number of features [3]. SGT is a stochastic gradient tree algorithm for incrementally constructing a decision tree using stochastic gradient information as the source of supervision. However, SGT mainly focuses on prediction performance rather than interpretability. What's more, the scalability of SGT is not clear for the numbers of features of datasets in its experiments are all less than 100 [4]. 

[2] Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems, pp. 3146–3154, 2017.
[3] Yang, Yongxin, Irene Garcia Morillo, and Timothy M. Hospedales. "Deep neural decision trees." arXiv preprint arXiv:1806.06988 (2018).
[4] Gouk, Henry, Bernhard Pfahringer, and Eibe Frank. "Stochastic Gradient Trees." In Asian Conference on Machine Learning (2019).

---

>Comment 5: Given that the model is trained with gradient descent, and arbitrary loss functions, could the model be evaluated on tasks other than classification?

Yes, it is easy to apply RRL to other tasks, e.g., regression. As future work, we plan to apply RRL to tasks like regression, data generation, etc., and evaluate the performance of RRL on these tasks.

From: ICLR.cc/2021/Conference/Paper1045/AnonReviewer5
title: Concerns with interpretability of the resulting model.
review: ## Summary
The authors propose a new approach for training interpretable discrete models via gradient descent. They claim three contributions: 1) incorporation of layers implementing logical conjunction and disjunction operations; 2) the gradient grafting technique for performing gradient descent on discrete structures; and 3) a logical activation function that enables models to scale to larger training datasets. Additionally, they also claim competitive performance with comparable learning methods.

## Strengths
* The paper is well organised and easy to follow.
* The proposed gradient grafting technique, while heuristic in nature, is novel and seems to work quite well in practice.
* The proposed method achieves good accuracy compared to the other approaches considered in the experiments.

## Weaknesses
* There is limited novelty in the proposed model. The continuous/discrete conjunction and disjunction operations used in the logical layers are the same as those used in Wang et al. (2020), however they are arranged differently in this method.
* The connection betweent the continuous version of the model and the discrete version is weak. One of the claimed contributions is that the discrete model is optimised directly, but this does not seem to be the case in practice---the continuous version of the model is optimised. It is also unclear how the continuous-valued weights are binarised in order to work with the discrete version of the model.
* The results in Figure 2 indicate that typically a large number of rules are learned, yielding models that are unlikely to be interpreted reliably. The log scale on the x axis of the plots is somewhat deceptive---I find it hard to believe that models with between 64--256 terms are interpretable. I think further investigating what tradeoff exists between accuracy and interpretability is necessary before the paper can be accepted.

## Other comments/questions
* Comparing average accuracy is generally not a good idea. It is more informative to compare the average rank of each method across datasets [1].
* Can the authors elaborate on how hyperparameters for each of the competing methods are chosen?
* The computational graph for gradient grafting seems related (yet distinct) to the Gumbel softmax estimator---perhaps the authors could discuss this relationship further?
* The authors could do more to discuss related work on training decision trees and rules using gradient-based optimisation. E.g., XGBoost [2], DNDT [3], and SGT [4].
* Given that the model is trained with gradient descent, and arbitrary loss functions, could the model be evaluated on tasks other than classification?

## Update
I thank the authors for some of the updates and response that address my concerns with clarity, but my main concern related to the interpretability aspect has not been resolved. The authors suggest that one can discard some of the rules learnt by the algorithm if the model becomes too large, using the weights in the linear layer to determine which rules should be kept. This seems like a good direction to explore, but it is unclear if a few rules typically dominate the predictions, or whether one will sacrifice significant accuracy in order to gain interpretability. As stated in my original review, I would have liked to see analysis of the *tradeoff* between interpretability and accuracy of this method.

A few other things:
* I still do not understand exactly how the hyperparameters were tuned. The authors have provided a list of those that were tuned, but I still don't understand the process used for tuning them.
* Some of the responses to other reviews have reinforced some of my concerns. In particular, it seems the distribution of weights attached to rules is not optimal for the proposed (but unevaluated) rule pruning method. Also, the authors conflate statistical significance with practical significance when replying to Reviewer 4.

[1] Demšar, Janez. "Statistical comparisons of classifiers over multiple data sets." Journal of Machine learning research (2006).
[2] Chen, Tianqi, and Carlos Guestrin. "Xgboost: A scalable tree boosting system." In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016).
[3] Yang, Yongxin, Irene Garcia Morillo, and Timothy M. Hospedales. "Deep neural decision trees." arXiv preprint arXiv:1806.06988 (2018).
[4] Gouk, Henry, Bernhard Pfahringer, and Eibe Frank. "Stochastic Gradient Trees." In Asian Conference on Machine Learning (2019).
rating: 5: Marginally below acceptance threshold
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: ICLR.cc/2021/Conference/Paper1045/AnonReviewer1
title: Good paper, accept
review: Summary:

This paper presents a new rule based  classifier called Rule based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation.  In order to train this model efficiently  the paper presents a learning algorithm that projects the discrete RRL model to a continuous space and hence optimise the model using gradient descent. Through experiments on 9 small and 4 large datasets shows that RRL improves over other methods, has low complexity and interpretable. 

Reasons for score:

I think this is a Good paper and vote for accepting.  This paper presents a solid contribution in learning rule based classifiers and hence to interpretable machine learning.  The paper is clearly written and superiority of the presented models are backed by strong experimental results.

Pros:

1. The paper is clearly written and easy to follow.
2.  The paper addressees an important problem of interpretable machine learning and presents a novel model which is scalable.
I find the technique of gradient grafting particularly interesting.
3.  Very strong experimental results and ablation studies.


rating: 7: Good paper, accept
confidence: 2: The reviewer is willing to defend the evaluation, but it is quite likely that the reviewer did not understand central parts of the paper

From: ICLR.cc/2021/Conference/Paper1045/AnonReviewer4
title: Good idea, but not enough impact.
review: Authors propose a new scalable classifier, named Rule-based Representation Learner (RRL), that can automatically learn interpretable rules for data representation and classification. For the particularity of RRL, the authors propose a new gradient-based discrete model training method, i.e., Gradient Grafting, that directly optimizes the discrete model. Authors also propose an improved design of logical activation functions to increase the scalability of RRL and make RRL capable of discretizing the continuous features end-to-end. The experimental results show good performance of RRL both high classification performance and low model complexity on data sets with different scales.

However, it is questionable whether the difference in accuracy between decision trees such as C4.5 and RRL in the experimental results is crucial. If the difference in accuracy is as small as this, the ease of interpretation of the decision tree may be superior to that of the RRL. Alternatively, the choice of experimental data may not be appropriate to demonstrate the superior performance of RRL.

I also didn't understand why the fuzzy/soft rule sacrifices the model interpretability. Could it be that the fuzzy representation is intended to be closer to human subjectivity, and thus aid in intuitive understanding?



rating: 5: Marginally below acceptance threshold
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: ICLR.cc/2021/Conference/Paper1045/AnonReviewer2
title: Multi-layer rule learner with missing discussions on scalability, interpretability and implementation details
review: The authors propose a classifier consisting of multiple layers. The inner layers construct rules in conjunctive normal form. The last layer is used to assign weights to the constructed rules. To train the overall model and obtain discrete solutions, the authors use a simple rounding mechanism leading to a method that they call gradient grafting. The paper concludes with a computational study, where the authors report the classification performance as well as the model complexity on a set of problems. 

This is an interesting paper proposing a new method to tackle the trade-off between interpretability and accuracy. Indeed, rule-based learners are considered to be more interpretable. The authors also claim that the proposed rule-learner is also scalable. Overall, introduction of a scalable, interpretable and accurate method could have been considered as a big achievement. However, I have several questions and comments about this achievement as I list below:

- Are the results given for test set? If so, what is the train-test percentage?

- What are the computation times? Without these figures, it is hard to assess the scalability of the proposed method.

- The numbers of edges that you report range from 50 to 1000. These values seem quite large. How does this affect the interpretability?

- In Figure 4, only five clear rules are reported. What is the distribution of the weights of the remaining rules?

-  How do you decide various design parameters; such as, number of layers (n_l), number of bins for feature discretization (k), number of layers (L)?

- As you need a binarization layer to divide the continuous features into bins, can't we just say that the method works only with discrete features? This is how it would be presented by other rule-learning methods.

- I guess Section 3.2 is superficial as it is straightforward to split a continuous feature into bins. Am I missing something here?

- How do your results compare against the results obtained with other rule/tree learning methods based on (integer) linear optimization? Some names from that field are Bertsimas,  Rudin, Gunluk.

- Can you guarantee that the resulting set of rules covers the entire feature space? In other words, is it possible that a test sample is not classified with the output set of rules?
rating: 6: Marginally above acceptance threshold
confidence: 5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature