Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The generalization of data distributions to unseen regions, i.e., extrapolation, remains one of the critical challenges in machine learning. The inductive biases of the learner determine such extrapolation. Unfortunately, machine learning systems often do not share the same inductive biases as humans and, as a result, may extrapolate in ways that do not match the analyst's expectations. The authors investigated two different types of such inductive bias: feature-level bias (differences in which features are more easily learned) and exemplar-based and rule-based bias (differences in how learned features are used for generalization). Inspired by these experimental approaches, we have proposed a protocol to investigate this trade-off in learning systems directly. We present empirical results for a range of models and the domains of explanatory images and language. We demonstrate that controlling for feature-level bias while measuring the trade-off between exemplars and rules provides a complete picture of extrapolative behavior than existing formalisms.
main_review: Generalization in the domain of extrapolation, which the authors address in this paper, is one of the critical issues in machine learning. Therefore, they propose a protocol to investigate the problem of inductive bias in extrapolation. Specifically, inspired by psychological research, they propose a protocol to investigate the inductive bias of the learning system towards different features (FLB) and the inductive bias of the learning system towards different ways of using features, either by rule-based or exemplar-based generalization (EVR).

The approach, inspired by psychological research, is interesting. However, we are not convinced that it is a practically valid method, as we have only shown experiments on two datasets.
summary_of_the_review: The authors' treatment of inductive bias in extrapolation is interesting and will interest many researchers. The proposed protocol, which takes its ideas from psychology, is also interesting. However, we are not convinced that the proposed protocol is practical.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes measures of how two types of explanation methods -- rule based and exemplar based -- generalize and extrapolate to unseen data regions.
main_review: Some of the ideas introduced by the paper to think about certain inductive biases is interesting. For example, feature-level bias being which features are easier or harder to learn. Here it could be interesting to go into continuous vs. categorical features.

I found some of the exposition unnecessarily confusing. For example, since in Section 2, the color of objects determines their label (i.e. green objects are “dax” and purple objects are “fep”), why use the labels "dax" and "fep" at all rather than just green and purple?

There are many claims that could be substantiated or explained further. For example, why do GPs help with formalizing exemplar-based generalization (Section 4.1)? It would also be clearer to specify which statements have been substantiated through experiments and which have not. For example, Section 4.2 starts off by saying that one NN, one GLM, and one GP has been trained and then make a jump to Figure 4a which shows decision boundaries that are not surprising; I am not sure how it validates the protocol proposed to measure EVR without confounds. Section 4.4 makes a substantial claim ("We also find that different ways to reduce ρ (e.g. by reducing π0 or by increasing π1), give different extrapolation behavior (see Appendix)") based on results pushed to the appendix. 

Some sentences could be better worded:
- (Page 6) "EVR increases with exemplar-basedness" 
- (Page 7) What is the "held-out quadrant"?

summary_of_the_review: The paper is ambitious but falls short in clearly explaining the framework proposed, and substantiating the idea with strong experiments.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes two measures for evaluating the feature-level bias and the exemplar-vs-rule bias in learning systems. Specifically, the authors designed three independent training conditions:
    i. cue conflict: {x1 = 0, x2 = 1, y = 0}, {x1 = 1, x2 = 0, y = 1}
    ii. zero shot: {x1 = 0, x2 = 0, y = 0}, {x1 = 1, x2 = 0, y = 1}
    iii. partial exposure: {x1 = 0, x2 = 0, y = 0}, {x1 = 1, x2 = 0, y = 1}, {x1 = 0, x2 = 1, y = 0}
and one testing condition:
    iv. extrapolation: {x1 = 1, x2 = 1, y = 1}.
The inductive bias of a given learning system is measured by its extrapolation performance difference when trained on different training conditions. 

Empirically, the authors first verified their framework on a synthetic dataset with 2D inputs. The results confirm that generalized linear model favors rule-based generalization while Gaussian process favors exemplar-based generalization. On IMDB, the authors show that LSTM models exhibit high feature-level bias (overfitting to the spurious token features) favors exemplar-rule bias. On CelebA, the authors show that ResNet exhibits a wide range of feature-level bias for different features (‘male’ is easier to learn than ‘high cheekbones’). However, across all feature pairs, the model prefers exemplar-based generalization to rule-based generalization.
main_review: Strength:
+ Probing the behavior of the learning system by comparing its generalization performance when trained on different datasets is interesting and novel.
+ The synthetic experiment on 2D inputs demonstrate the effectiveness of the measure. 
+ The paper is very well-written.

Weakness:
+ Unclear utility.
The main contributions of the paper are the two proposed measures. However, I found it hard to utilize these measures in practical applications. 
  + In order to compute the measures, the values for the discriminant features and the distractor features are required. In this situation, there are many existing methods for learning models that are robust against the distractor features [1]. What will be the point of evaluating the bias of a non-robust classifier?
  + For the exemplar-vs-rule propensity (EVR), my understanding is that it captures the extent to which the distractor features are utilized in the learning system. What will be the benefit of EVR compared to evaluating the worst-group accuracy as in [1]? Let’s use CelebA as an example. The training data contains mostly {male, dark_hair}, {female, blond_hair}, {female, dark_hair} images. At test time, [1] evaluates the worst-group accuracy, which is likely to be the performance on {male, blond_hair} (since it is underrepresented in the training data).

Questions and suggestions:
+ How do you interpret the values of the FLB and EVR measures? For instance, in IMDB, the FLB score is -0.3 while the EVR score is 0.54. What do these values mean?
+ I think both measures can be defined in a more generalized setting (where you don’t need to assume the label marginal is uniform).
+ You argued in the paper that the measure is capturing something more than the spurious correlation. I think it will be more precise to state it as **linear** correlation in Figure 3.
+ It will be very interesting to see a contour plot of EVR over $\pi_0$ and $\pi_1$ even in the synthetic experiments.

I am happy to adjust my ratings after the rebuttal period.

[1] Sagawa, Shiori, et al. "Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization." arXiv preprint arXiv:1911.08731 (2019).
summary_of_the_review: The authors proposed an interesting approach for measuring the rule- and exemplar-based  generalization for a given learning system. The perspective is novel and the writing is clear. My only concern is the practical utility of the proposed measures.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper studies the extrapolation of machine learning models to unseen regions, and specifically studies two types of biases: feature-level bias (differences in which features are more readily learned) and exemplar-vs-rule bias (differences in how these learned features are used for generalization). Motivated by the studies of exemplar vs. rule-based generalization in cognitive psychology, the authors present a protocol directly probing this trade-off in machine learning systems. The authors present empirical results across a range of models in both expository and real-world image and language domains and demonstrate that using the trad off provides a more complete picture of extrapolation behaviour than existing methods. 

main_review: I found that this paper was quite difficult to read since the authors present discussions without clear definitions. Using many acronyms makes the reading difficult too. 

For example, “feature-level bias is measured as deviation from chance performance in the CC condition.” When reading CC condition, it says “the data presented in this condition confound color and shape”. What does confound mean here? CC condition is unclear and I do not know feature-level bias.

Similarly, “Exemplar-vs-rule bias is measured by the difference between performance in the PE and ZS conditions—…” When reading PE and ZS, they are not defined either. 

Reading on, I do not see formal definitions of feature-level bias exemplar-vs-rule bias. There are measures obtained from models trained by some specific methods. So the proposed approach deals specifically with three types of methods. Why do the authors focus on three types of methods? Why are they representative? 

Also, I do not know what spurious correlation means in this paper. 

The authors present quite some examples to illustrate their points, but I am confused by their examples. For example, in Figure 1, only two training examples are given. Any prediction is incorrect since there is not enough evidence to learn a model. I cannot see the differences between rule-based and example-based. What rules can we generate based on the two examples?

summary_of_the_review: The paper is unclear based on the current presentation.
review_title: null

Meta-review: The paper proposes a novel protocol for examining the inductive biases in learning systems, by quantifying the exemplar-rule trade-off (as measured by the exemplar-vs-rule propensity (EVR) defined in Eq. (2)) while controlling for feature-level bias. 

Reviewers mostly agree that the problem studied in this paper is practically relevant and that the two bias measures are potentially interesting and (jointly) more informative than existing measures such as spurious correlation. However, a shared concern among the reviewers (with confidences scores >=3) is the clarity of the exposition (e.g., many key concepts such as the data conditions are informally specified [Section 2 (Reviewer TPBn)], some key messages not clearly conveyed in the main paper [Section 3 (Reviewer RJtk)], and results inconclusive or not sufficiently supported by the experimental results [for both the synthetic setting (Reviewer RJtk) and the real-world setting (Reviewer yoH5)]. Based on the above concerns, the reviewers were not convinced that this work is well supported in its current state to merit acceptance for publication.
Decision: Reject