Paper: Interactive Model Correction with Natural Language (ICLR-2024)

Reviewer comment (Presentation):
Paper may be improved from better organization. For example, I don't see a strong motivation that the related work section is between the method and the experiments.

Author response:
Thank you for the suggestion; we have moved the related work section to right after the introduction. Our original thought was that we may want to jump into new things in this paper (section 4) sooner since our intro and related work are long, but we agree that this organization makes the flow from method to experiments smoother.

Reviewer comment (Experiments):
More information is needed for the baselines and results. For example, how is "class-balanced"/"wort-class"/"slice-balanced"/"worse-slice" defined? Also what is DFR and how is Clarify different from them? Such information is necessary for me to evaluate the contributions of Clarify.

Author response:
Thank you for pointing out potential points of confusion. We have added the information below to the paper:
- DFR: Deep Feature Reweighting, Kirichenko et al "Last layer re-training is sufficient for robustness to spurious correlations." This differs from the reweighting of Clarify in that DFR requires datapoint-wise group annotations, whereas Clarify only requires one text description of the failure mode.
- class-balanced: ERM loss, balanced so that each class has equal weight.
- worst-class: ERM loss applied to the class with the highest average loss.
- slice-balanced: ERM loss, balanced so that each slice has equal weight.
- worst-slice: ERM loss applied to the slice with the highest average loss per class. Here, "slice" is determined by the annotation from Clarify.

Reviewer comment (Experiments):
Comparison with automated methods is not convincing. Table 2 presents comparison with zero-shot/prompted models, while Clarify is a fine-tuning based method. Clarify outperforming RoboShot is not a convincing evidence. Comparison with fine-tuning based spurious correlation mitigation method is needed.

Author response:
Table 1 compares to representative fine-tuning methods for mitigating spurious correlations, including various variants of ERM, DFR, and Group DRO. The comparison to RoboShot was meant to demonstrate that Clarify is significantly more effective among methods based on text. We agree that this comparison is not apples-to-apples due to the difference in data assumptions; we have moved this comparison to the appendix to avoid giving readers the wrong impression.

Reviewer comment (Theory):
What are the design considerations behind the error score in equation 2?

Author response:
As we are eliciting features that separate correct examples from incorrect ones, we wanted to provide a measure of that. We use class-balanced accuracy to account for scenarios where the number of correct and incorrect examples are different. We have added a bit more discussion to the error score definition.

Reviewer comment (Experiments):
Currently the method involves re-weighting the training set and re-training the model. Is it possible to use the initially trained model and fix the spurious correlations by further training it?

Author response:
Interesting suggestion, we tried further training (i.e. fine-tuning) with text feedback from Clarify in the CLIP ResNet-50 setting (Table 1). We show worst-group accuracies on Waterbirds and CelebA below:

| Method | Waterbirds | CelebA |
|:------|:-------------:|:---------:|
| ERM | 63.4 | 31.1 |
| Clarify | 75.7 | **89.1** |
| Clarify (fine-tuning) | **75.8** | 88.9 |

Clarify fine-tuning significantly outperforms the naively trained ERM model while showing very similar performance to simply re-training with Clarify. We think this is because the learning objective induced by our text feedback discourages the use of spurious correlations enough that the starting parameters have little effect on the final model.

Reviewer comment (Evaluation):
How is the "7.3% on average" computed?

Author response:
We identified 31 ImageNet classes with hard subpopulations; the average accuracy on these hard subpopulations is 7.3% higher after reweighting with Clarify.

Reviewer comment (Novelty):
Missing reference: https://arxiv.org/abs/2210.00055

Author response:
Thank you for pointing these out; we were not aware of these and have added them to our related work section.

Reviewer comment (Experiments):
**Role of initial starting point**. CLARIFY shows some keywords as an initial point for further human response. Given the risk with random phrases (getting high error scores) as explained above, I believe the controlled language that the annotators are initially presented with has much role to play than what is emphasized in the paper. Please address related questions to establish their role.

Author response:
We do __not__ provide any initial keywords for the user studies in section 5.2 (figures 3, 4, 5). The users came up with all the prompts on their own. Our supplementary video shows exactly what information was provided to these users. We edited the paper to make the setup of the user study clearer. We used automated keyword generation only for the ImageNet experiment, where their primary role was to identify the most promising classes among the 1000.

Reviewer comment (Experiments):
**Elicitation is too simple even for the simple tasks.** The expert phrases for WaterBird and CelebA are in combination of the label, i.e. forest × water/land bird, man × blonde/non-blonde for WaterBirds and CelebA dataset respectively. CLARIFY only elicits the keyword and not their combination, thereby missing on the true compositional phrase. This is somewhat of a minor issue and goes to only show the difficulty in describing failure modes using language.

Author response:
Our problem formulation is class-conditional, so it is not necessary to include the class in the description. For example, when we collect phrases for the waterbird class, the phrase "forest" naturally refers to images of waterbirds in a forest.

To your general point, while we agree that CLIP likely struggles to understand descriptions of compositional or otherwise complex failure modes, we do not think this is a fundamental limitation of our proposed framework. We expect future multimodal models to be able to understand increasingly more complex descriptions, and we think the bottleneck will be in efficiently eliciting these descriptions from humans. We note that Clarify automatically bridges from elicited descriptions to model improvements, so any future improvement in the backbone multimodal model brings with it, out of the box, an expanded set of describable failure modes. We have added discussion around this limitation and future work to section 6.

Reviewer comment (Experiments):
**Role of humans**. The experiments did not paint a convincing role of humans in specifying the error pattern. Humans are not necessarily good at finding common patterns across many misclassified examples. Besides, no single pattern or keyword may explain the many miscalssfied examples. Please justify the required human skill and the potentially poor payoff for their effort (i.e. performance payoff for each keyword may not worth the effort).

Author response:
While we agree that finding patterns in misclassified examples is non-trivial, our user study shows that we can effectively operationalize this task. Specifically, given an interface juxtaposing two grids of correct and incorrect images side-by-side, non-experts with no prior experience could identify failure modes at an average pace of 2.7 minutes per dataset. Prior works in the HCI literature show that crowds of people excel at generating ideas [1,2], particularly when asked to compare and contrast two cases to extract high-level differences [3,4].

We believe that when identifying failure modes, including humans in the loop is crucial. First, we note that even automated methods ultimately require human oversight to validate that identified keywords genuinely represent significant subpopulations. Our research recognizes this dependency and offers an efficient framework for eliciting these descriptions from humans. Second, as our user study demonstrates, non-expert users are able to identify features that are uncommon phrases or are otherwise not easily detected, such as “bleach blonde” or “dirty blonde.” Finally, even if a given keyword doesn’t end up improving performance in an immediately measurable way, it provides useful feedback to the model developer as a fine-grained audit/evaluation of model behavior that isn’t possible through standard validation sets. This can result in efforts to monitor the performance of error-prone subgroups or even just help the model developer be aware of the non-resolvable limitations of the model.

[1] Yu and Nickerson, Cooks or cobblers? crowd creativity through combination (CHI 2011).

[2] Kim, Cheng, and Bernstein, Ensemble: Exploring Complementary Strengths of Leaders and Crowds in Creative Collaboration (CSCW 2014)

[3] Fogarty et al, CueFlik: interactive concept learning in image search (CHI 2008).

[4] Cheng and Bernstein, Crowd-Machine Learning Classifiers (CSCW 2015).

Reviewer comment (Novelty):
Kim and Mo et.al. (Bias-to-Text: Debiasing Unknown Visual Biases through Language Interpretation) that is mentioned in the paper proposed an automated discovery of keywords in the same setting as CLARIFY but without the need for human in the loop. They identified the expert keywords for both CelebA and Waterbirds and even demonstrated some results on ImageNet variants. More elaborate discussion and comparison with Kim and Mo et.al. is expected especially since identifying keywords is not easy for humans.

Author response:
Indeed, Bias-to-Text is closely related as it extracts keywords using the difference in CLIP embeddings of correct vs incorrect images. We have added an experiment to the paper in which we evaluate Bias-to-Text on the ImageNet classes for which we have annotations through Clarify. The experiment in Table 2 and Appendix B show that both the top 10 keywords and the suggested ranking are not reliable: the top suggested keyword is never the one we identified through Clarify, and ours is only included in their top 10 in 5/31 classes. This suggests that in its current form, automated bias discovery methods ultimately require human oversight to identify the most relevant bias descriptions. We view these two research directions as complementary: automated methods assist human-in-the-loop interfaces, which in turn can guide the design of automated methods. We have added this discussion to our appendix.

Reviewer comment (Presentation):
What are the keywords presented to the user in Fig. 3 results?

Author response:
We do __not__ provide any initial keywords for the user studies in section 5.2 (figures 3, 4, 5). The users came up with all the prompts on their own. Our supplementary video shows exactly what information was provided to these users. We edited the paper to make the setup of the user study clearer. We used automated keyword generation only for the ImageNet experiment, where their primary role was to identify the most promising classes among the 1000.

Reviewer comment (Evaluation):
The top-6 phrases picked on CelebA are all outlier or random features such as *darker blonde, darker than..., any other...*. Even on WaterBirds dataset in Figure 3, we see somewhat random phrases such as *ducks, waterfowl* getting good error score. Why is that? On CelebA, *darker blonde*, *dirty blonde* resemble outlier features that may have been rare in the training dataset. How can CLARIFY prevent specification of outlier or random features over spurious features?

Author response:
We believe that outlier features are useful in a similar way to spurious correlations: they identify a specific subpopulation that standard training fails to capture. As we do for spurious correlations, we can adjust the training procedure to account for these failure modes, e.g. by reweighting or collecting more data.

Reviewer comment (Experiments):
Please also present results or your comments on what happened when the user-study is conducted without the initial nudge of keywords?

Author response:
We do __not__ provide any initial keywords for the user studies in section 5.2 (figures 3, 4, 5). The users came up with all the prompts on their own. Our supplementary video shows exactly what information was provided to these users. We edited the paper to make the setup of the user study clearer. We used automated keyword generation only for the ImageNet experiment, where their primary role was to identify the most promising classes among the 1000.