Learning Human-Compatible Representations for Case-Based Decision Support
Authors: Han Liu
Yizhou Tian
Chacha Chen
Shi Feng
Yuxin Chen
Chenhao Tan
Venue: ICLR 2023 poster (2023)
Keywords: human-compatible representation learning
human triplet judgments

Abstract: Algorithmic case-based decision support provides examples to help human make sense of predicted labels and aid human in decision-making tasks. Despite the promising performance of supervised learning, representations learned by supervised models may not align well with human intuitions: what models consider as similar examples can be perceived as distinct by humans. As a result, they have limited effectiveness in case-based decision support. In this work, we incorporate ideas from metric learning with supervised learning to examine the importance of alignment for effective decision support. In addition to instance-level labels, we use human-provided triplet judgments to learn human-compatible decision-focused representations. Using both synthetic data and human subject experiments in multiple classification tasks, we demonstrate that such representation is better aligned with human perception than representation solely optimized for classification. Human-compatible representations identify nearest neighbors that are perceived as more similar by humans and allow humans to make more accurate predictions, leading to substantial improvements in human decision accuracies (17.8% in butterfly vs. moth classification and 13.2% in pneumonia classification).

Published as a conference paper at ICLR 2023
LEARNING HUMAN-COMPATIBLE REPRESENTATIONS FOR CASE-BASED DECISION SUPPORT
Han Liu, Yizhou Tian, Chacha Chen, Shi Feng, Yuxin Chen & Chenhao Tan Department of Computer Science, University of Chicago (hanliu, iu, t tianh, chacha, shif, chenyuxin, chenhao) Buchi cago.
ABSTRACT
Algorithmic case- uuse-based decision support provides examples to aid people in de- cision making tasks by providing contexts for test case. Despite the promising performance of supervised learning, representations learned by supervised models may not align well with human intuitions: what models consider similar examples can be perceived as distinct by humans, As a result, they have limited effectiveness in case-based decision support. In this work, we incorporate ideas from metric learning with supervised learning to examine the importance of alignment for effec- human-provideds provided triplet judgments to learn human-compatible decision-focused representations. Us- ing both synthetic data and human subject experiments in multiple classification tasks, we demonstrate that such representation is better aligned with human per- ception than representation solely optimized for classification. Human-compa compatible representations identify nearest neighbors that are perceived as more similar by humans and allow humans to make more accurate predictions, leading to sub- stantial improvements in human decision accuracies (17.8% % in butterfly vs. moth classification and 13.2% pneumonia classífication).
tive decision support. In addition to instance-level labels, we human-
INTRODUCTION
Despite the impressive performance of machine learning (ML) models, humans are often the final decision maker in high-stake domains due to ethical and legal concerns (Lai & Tan 2019 Green & Chen 2019), so MI. models as decision support is preferred over full automation. In order to provide meaningful information to human decision makers, the model cannot be illiterate in the underlying problem, e.g., model for assisting breast cancer radiologists should have high diagnostic accuracy by itself. However, model with high autonomous performance may not provide the most effective decision support, because it could solve the problem in a way that is not comprehensible or even perceptible humans, e.g., AlphaGo's 's famous move 37 (Silver et al.) 2016 2017 Metz et al.) 2016). Our work studies the relation between these two objectives that effective decision support must balance:
achieving high autonomous performance and aligning with human intuitions.
We focus on case- decision support for classification problems (Kolodneer 1991 Begum et al.) 2009 Liao| 2000 Lai & Tan 2019). For each test example, in addition showing the model's predicted label, case-based decision support shows one or more related examples retrieved from the training set. These examples can be used to justify the model's prediction, e.g., by showing similar-looking examples with the predicted label, or to help human decision makers calibrate its uncertainty, e.g., by showing lookang examples from other classes. Both use cases require the model to know what is similiar-looking to the human decision maker. In other words, an important consideration in aligning with human intuition is approximating human judgment of similarity.
Figure T illlstraate the importance of such alignment on classification problem of distinguishing butterfly from moth. A high-accuracy ResNet (He et al., (2066) produces a highly linearly-separable representation space, which leads to high classification accuracy. But the nearest neighbor cannot provide effective justification for model prediction because it looks dissimilar to the test example for humans. The similarity measured in model representation space does not align with human visual similarity. If we instead use representations from a second model trained specifically to mimic human visual similarity rather than to classify images, the nearest neighbor would provide strong

---

Published as a conference paper at ICLR 2023
Test Image
Human Perception Nearest Neighhor
Math Buttetly
ResNet
Nearest Neighbor
Figure 1: Nearest neighbor retrieved by the model representation might align with human similarity judgment. The MLE representations (5 (512-dim) are visualized using t-SNE (Van der Maaten Hinton 2008). The purple circle represents a specific test instance, The nearest neighbor found by MLE representations (pink circle) is not as visually similar as the instance in cyan circle found by optimizing a metric learning objective.
justification for the model prediction. However, using the second model for decision support has the risk of misleading or even deceiving the human decision maker because the "justification" is generated based on representation space that is different from the model used predict the label; it becomes persuasion rather than justification.
The goal of this work is to learn single representation space that satisfies two properties: (i) pro- ducing easily separable representations for different classes to support accurate classification, and (ii) constituting metric space that is aligned with human perception of similarity between exam- ples. Simultancously matching the best model on classification accuracy and achieving perfect approximation of human similarity might not be possible, but we hypothesize that good trade-off between the two would benefit decision support. We propose a novel multi-task learning method that combines supervised learning and metric learning. We supplement the standard maximum likelihood objective with a triplet margin loss function from Baintas et al. (0166). Our method learns from human annotations of similarity judgments among data instances in the triplet form.
We validate our approach with both synthetic data and user study. We show that representations learned from our framework identify nearest neighbors that are perceived as more similar by the synthetic human than that based on supervised classification (henceforth MLE representations, see $2 for details), and are therefore more suitable to provide decision support. We further demonstrate that the advantage of human-compatible representations indeed derives from human perception rather than data augmentation.
We further conduct human subject experiments using two classification tasks: (i) butterfly vs. moth classification from ImageNet (Krizhevsky et al., 2012), and (ii) pneumonia classification based on chest X- X-rays (Kermany 2018). Our results show that human-come comanncompaaible representations provide more effective decision support than MLE representations. representations allow laypeople to achieve accuracy of 79.1 in pneumonia classification, 15.3% higher than MLE representations. A similar improvement has been observed on the butterfly vs. moth classification task (34.8% over MLE representations and 17.8% over random).
particular, numan-compatiin
To summarize, our main contributions include:
We highlight the importance of alignment in learning human-compatible representations for case- based decision support.
We propose multi-task learning framework that combines supervised learning and metric learning to simultaneously learn classification and human visual similarity.
We design novel evaluation framework for comparing representations in decision support. Empirical results with synthetic data and human subject experiments demonstrate the effectiveness of our approach.
2

---

Published as a conference paper at ICLR 2023
2 CASE-BASED DECISION SUPPORT
Consider the problem of using classification model h X - - decision support for humans. Simply showing the predicted label from the model provides limited information. Explanations are commonly hypothesized to improve human performance by providing additional information
Delshi-Velez & Kim 2017). We focus on information presented in the form of examples from the
training data, also known as case-based decision support (Kolodneer [991] Begum al. 2009 Liao
2000; Lai & Tan 2019). Case-based decision support can have diverse use cases and goals. Given a test example (Ir) and its predicted label (ÿ), two common use cases are:
Presenting the nearest neighbor of along with label ÿ as a justification of the predicted label. We refer to this scenario as justification (Kolodneer) 1991).
Presenting the nearest neighbor in each class without presenting ÿ. This approach makes a best- effort attempt to provide evidence and leaves the final decision to humans, without biasing humans
with the predicted label. We refer to this scenario neutral decision support (Lai & Tan 2019).
Formulation. Building on Kolodneer (1991), we formalize the problem of case-based decision support in the context of representation learning. The goal is to assist humans on classification problem with groundtruth f X - V. We assume access to a representation model g, which takes an input X and generates m mn-dimensional representation g(x) E R For each test instance r, an example selection policy chooses k labeled examples from the training set Diaain and shows them to the human (optionally along with the labels); the human then makes a prediction by choosing a label from V. As discussed in the two common use cases, we consider nearest- wewrresttnigghoo-biieee selection policies in this work. The focus of this work thus on the effectiveness of g for case-based decision support.
Given a neural classification model h : X' + D, the representation model is the last layer before the classification head, which is a byproduct derived from h. We refer to this model as e(h) In justification, the example selection policy is π = NN(x, ( where ğ = h(x)) Dirain refers to the subset of training data with label 9 (i.e.. {(x, y) E Durain y = o}), and NN finds the nearest neighbor of x using representations from e(h) among the subset of examples with label g. In decision support, the example selection policy is (NN(x, e(h). Dirai), Vy E V).
Misalignment with human similarity metric is detrimental. We argue that aligning model representations with human similarity metric is crucial for case-based decision support; we refer to it as the metric alignment problem. To illustrate the importance of alignment, we need to reason about the goal of case-based decision support. Let us start with justification, which is a relatively easy case. To justify a predicted label, the chosen example should ideally appear similar to the test image. Crucially, this similarity is perceived by humans (i.e., interpretable), and the example selection policy identifies the nearest neighbor based on model representation (i.e., faithful). The gap between human representation and model representation (Fig. leads to undesirable justification.
Neutral decision support, however, represents a more complicated scenario. We start by emphasizing that the goal is not simply to maximize human decision accuracy, because may use policies that intentionally show distant examples to nudge or manipulate humans towards making a particular decision
P Choosing the nearest neighbors in each class is thus an attempt to present faithful and neutral evidence from the representation space so that humans can make their own decisions, hence preserving their agency. Therefore, the chosen nearest neighbors should be visually similar the test instance by human perception, again highlighting the potential gap between model representation and human representation. Assuming that humans follow the natural strategy by picking the presented instance that's most similar to the test instance and answering with the corresponding label, then ideally, nearest neighbors in each class retain key information useful for classification so that they can reveal the separation learned in the model.
is unlikely that we get high alignment by solely optimizing classification even when the model's classification accuracy is comparable to the human's. Models trained with supervised learning
In general, we can use the representation any layer, but in preliminary experiments, we find representation from the last layer is most effective.
We will consider one such policy for the sake of evaluating the quality of representations in
3

---

Published as a conference paper at ICLR 2023
almost always exploit patterns in the training data that are (i) not robust to distribution shifts, and (ii) counterintuitive or even unobservable for humans (lyyss et al. 2019; Xiao et al., (020).
Combining metric learning on human triplets with supervised classification. We propose to address the metric alignment problem with additional supervision on the human similarity metric. We collect data in the form of human similarity judgment triplets (or triplets for short). Each triplet is an ordered tuple: (x" "r'..tt_ x which indicates x is judged by humans as being closer to the reference x' than (Balntas al. 2016). Given a triplet dataset T and labeled classification dataset D, learn a model / using triplet margin loss (Balntas et al. (016)) in conjunction with cross-entropy loss, controlled by a hyperparameter A:
A
log (po(ulz)) (a.a)~>
+(1-))
Σ
max (do(a" - -d do(x", 1,0)
(I)
(ar )u
entropy loss
Triplet margin loss
where d:(:s.) is the similarity metric based on model representations: we use Fuclidean distance. In this work, we initialize A with a pretrained ResNet (He al., 2016). When À and the triplet margin loss is turned off, the model reduces to a finetuned ResNet. When À = 0 and the cross-entropy loss is turned off, the model reduces to the triplet based-learning model [Balntas et al. (2016); we call it TMLMode 1 and will use it to simulate humans in some synthetic experiments in the appendix. Our work concerned with the representations learned by these models. Our approach uses the representations learned with X = 0.5 (henceforth human-com compatible representations and HC for short). We refer to the representations fine-tuning ResNet with the cross-entropy loss as MIE representations (MLE. for short) and the representations from TMIModel as TML.
3 EXPERIMENTAL SETUP
In this section, we provide the specific model instantiation and detailed experiment setup.
Models. All models and baselines use ResNet-18 18 (He et al., 2016) pretrained on ImageNet as the backbone image encoder. Following [Chen et al. (2020), we take the output of the average pooling layer and feed it into an MLP projection head with desired embedding dimension. We use the output of the projection head as our final embeddings (i.e., representations), where we add task-specific head and loss for training and evaluation. We use Euclidean distance the similarity metric for both loss calculation and distance measurement during example selection in decision support.
Our first baseline uses representations from ResNet finetuned with classification labels using cross- entropy loss (i.c., MLE). ResNet typically achieves high classification accuracy but does not necessarily produce human-aligned representations. Our second baseline uses representations from the same pretrained model finetuned with human triplets using triplet margin loss (Balntas et al. (016)) (i.e., TML). We expect TML produce more aligned representations but achieve lower classification accuracy than MLE and may provide limited effectiveness in decision support.
Our representations, HC, are learned by combining the loss terms following Equation [ The hyperparameter λ controls the trade-off between metric alignment and classification accuracy: with higher À we expect HC be more similar to MLE, while lower À steers HC towards TML. Empirically tuning À confirms this hypothesis. For the main paper, we present results with À = 0.5. More details about model specification and hyperparameter tuning can be found in the appendix.
wew.icatttiiiiinnssseeeee triplets. Human triplets may not always align with classification: triplet annotators may choose the candidate from the incorrect class over the one from the correct class. We refer to these data points as classification- cuassfication-incossstttr inconsistent triplets. We consider variant of human- compatible representations where we isolate human intuition that's compatible with classification and remove these ieassfficationnnconsssttte as oi-filteeedd Filtering is yet another way to strike a balance between human intuition and classification. We leave further details on filtering in the appendix.
Filtering
triplets from the training set; refer to this condition
Evaluation metrics. Our method is designed to align representations with human similarity metrics and at the same time retain the representations' predictive power for classification. We can evaluate these representations with classification and triplet accuracy using existing data, but our main evaluation is designed to simulate base- decision support scenarios.
4

---

Published as a conference paper at ICLR 2023
Negpolo
Nopulle2
VW dataset. (a) shows the dataset where labels are determined (non-linearly) by two features: the head and the body size of the fictional insects. (b)-(d) show samples of the two classes; the Weevil has a mid-sized body and mid-sized head, while the Vespula does not. Tail length and texture non-informative features.
Figure 2:
Head-to-head comparisons ("H2H"). To evaluate justification, we set up head-to-head compar- isons between two representations (Ri vs. R2) and ask: given test instance and two justifications retrieved by Rí and R2, which justification do humans consider as closer to the test instance? We report the fraction of rounds that R1 is preferable. In addition the typical justification for the predicted label, we also examine that for classes other than the predicted class, as those examples will be used decision support for users examine the plausibility of each class. We refer the nearest example in the predicted class as NI, and the nearest example in the other class as NO.
Neutral decision support. Following $2 we retrieve the nearest neighbors from each class. We the accuracy of humans the measure of effective decision support.
Persuasive decision support. We retrieve the nearest example with the predicted label and the furthest example from the other class. If the representation is aligned with human similarity metric, this approach encourages people to follow the predicted label, which likely leads to over-reliance and may unethical practice. Here, we use this scenario as a surrogate to evaluate the quality of the learned representations.
Note that we do not show model predictions so that humans focus on the similarity between examples.
4 SYNTHETIC EXPERIMENT
To understand the strengths and limitations of our method, we first experiment with synthetic datasets. Using simulated human similarity metrics, we control and vary the level disagreement between the classification groundtruth and the synthetic human's knowledge.
4.1 SYNTHETIC DATASET AND SIMULATED HUMAN SIMILARITY METRICS
We use the synthetic dataset "Vespula Weevil" (VW) from Chen al. (0188). It is binary image classification dataset of two fictional species of insects. Each example contains four features, two of texture-are are completely non non- -reediccii We generate 2000 images and randomly split the dataset into training. validation, and testing sets in a 60%:20%:2000 ratio. The labels are determined by various synthetic decision boundaries, such as the one shown in Fig. 2h.
and body size-are predictive of the label, and the other two-tail length and
To generate triplets data, we define simulated human similarity metrics weighted Euclidean distance over the visual features: for any instance a and b, b(a,b) VE, w,(a, 62)2, where i refers to the i-th feature. By changing the weight of each feature, we can control the level of disagreement between a synthetic human and the groundtruth. All procedures that involve humans (i.e., triplet data collection and evaluation) are done by the synthetic human in this section.
To quantify the disagreement, we use 1-NN classification accuracy following the synthetic human similarity metrie; we refer to it as the task alignment score. Note that this is different from our main alignment problem, which about the representations. The task alignment score ranges from 50% (setting the informative features' weights to 0 and distractor weights to 1) to 100%. See the appendix for more details on how we generate these weights. In each setting, we generate 40,000 triplets.
4.2 RESULTS
We compare HC, MLE, TML on classification accuracy, triplet accuracy, and decision support perfor- mance for the synthetic human. We train all three representations with large dimension of 512 and
5

---

Published as a conference paper at ICLR 2023
Table I: Experiment results on VW with H2H comparison and decision support evaluations.
Task alignment
50%
80%
83%
92%
92.5%
100%
Weights
[0,0,1,1]
[1,0,1,1]
[0,1,1,1]
[1,256,256,266]
[1,1,1,1]
NI-H2H
HC vs. MLB
0.917
0.914
0.903
0.880
0.872
0.808
NO-H2H
HC vs. MLE
0.916
0.968
0.946
0.958
0.962
0.970
Neutral decision support
MLE
0.753
0.899
0.896
0.897
0.901
0.929
TML
0.568
0.775
0.807
0.868
0.877
1.000
BC
0.759
0.901
0.928
0.949
0.955
1.000
Persuasive decision support
MLE 0.704
0.900
0.903
0.903
0.901
0,919
TML
0,906
0.881
0.863
0.876
0.877
1.000
HC
1.000
1.000
1.000
1.000
1.000
1.000
small dimension of 50 and observe that the 512-dimension representation is preferable based most metrics. We also train HC on filtered vs. unfiltered triplets as well as with different values A. For our main results, we report the performance with À 0.5 and filtered triplets for the decision boundary in Fig. 2a. We will discuss the effect of filtering later in this section. X's role is relatively limited and we will discuss its effect and other decision boundaries in the appendix.
In synthetic experiments, HC achieves the same perfect classification accuracy as MLE (100%), and triplet accuracy of 96.8%, which is comparable to TML (97.3%). This shows that HC indeed learns both the classification task and human similarity prediction task. We next present the evaluation of case-based decision support with the synthetic human, which is the key goal of this work.
HC significantly outperforms MLE in H2H. If there is no difference between HC and MLE, the synthetic human should prefer HC about 50% of times. However, as shown Table [ our synthetic human prefer HC over MLE by a large margin (about 90% of times) as justifications for both nearest in-class examples and nearest out-of-class examples, indicating the NIs and NOs selected based on the HC representations are more aligned with the synthetic human than MLE. For NI H2H, the preference towards HC declines as the task alignment improves, because if alignment between human similarity and classification increases, MLE capture human similarity as a byproduct of classification.
HC provides the best decision support. Table I shows that HC
achieves the highest neutral and persuasive decision support curacies in all task alignments. In neutral decision support, MLE consistently outperforms TMT,, highlighting that representation solely learned for metric learning is ineffective for decision support. For all models, the decision support performance im- proves as the task alignment increases, suggesting that decision support easier when human similarity judgment is aligned with the classification task. MLE and TML are more comparable persuasive decision support, while HC consistently achieves 100%. The fact that MLE shows comparable performance be- tween neutral and persuasive decision support further confirms that MLE does not capture human similarity for examples from different classes.
Neatral Acc
0.9
0.7
0.5
50.0
80.0
K3.0
92.0
92.5
100.
Alignment (5)
Figure 3: Neutral decision sup- port with HC and HC-fil tered. HC-filtered leads to improved performance.
Filtering triplets leads to better decision support. Fig. 3 shows that filtering glass-inconsistenn triplets improves HC's decision support performance across all alignments. Further details in the appendix show that filtering slightly hurts H2H performance. This suggests that in terms of decision support, the benefit of filtering out human noise may outweigh the loss of some similarity judgment.
The importance of human perception. One may question whether filtering class-inconsistent triplets essentially provides additional label supervision in the form of triplets. We show this is not the

---

Published as a conference paper at ICLR 2023
Table 2: Experiment results on VW using synthetic human with 92% alignment. Comparing MLE representations and HC-filtered with HC trained on label-derived triplets and HC trained on same-class triplets. 40,000 new triplets were generated for each condition.
Evaluations
MLE
HC label-derived triplets
HC same-class triplets
HC uC-filtered
NI-H2H with MLE
N/A
0.509
0.890
0.889
NO-H2H with MLE
N/A
0.607
0,970
0,958
Neutral DS
0.897
0.723
0,960
0,949
Persuasive DS
0.903
0.803
0.998
1.000
case by experimenting with HC trained on label-derived triplets. Assuming that an instance is more similar to another instance with the same label than one with a different label, we derive label-derived triplets directly from groundtruth labels (x: from the same class as x" and x from the other class), containing no human perception information. Table 2 shows decision support results for this setting: HC label-derived triplets show worse performance than HC- soc-filted filtered. In fact, HC label-derived triplets show even worse neutral and persuasive decision support than MLE, which may be due to label-derived triplets causing overfitting. This suggests that triplets without human perception do not lead to human-compatible representations.
We also experiment with trained on same-class triplets, human-triplets but only those where the non-reference cases (x+ x=) are from the same class; that is, the triplets cannot provide any label supervision. We observe from Table that HC trained on these triplets show similar results to So-filtered across all decision support evaluations. This suggests that human perception is the main factor in driving human- compaill representations' high decision support performance.
5 HUMAN SUBJECT EXPERIMENTS
We conduct human subject experiments on two image classification datasets: natural image dataset, Butterflies v.s. Moths (BM) and a medical image dataset of chest X-rays (CXR). For BM, we followed[Single al. (2014) and acquired 200 images from ImageNet (Krizhevsky et all (012). BM is a binary classification problem and each class contains two species. CXR is a balanced binary classification subset taken from Kermany al. (2018) with 3,166 chest X-ray images that are labeled with either normal or pneumonia. We randomly split the datasets following 60%:20%: 200% ratio. The classification accuracy with our base supervised learning models are 97.5% for BM and 97.3% for CXR. We only present results with human subjects in the main paper, but results from simulation experiments with TML, as synthetic agent, such as filtering triplets providing better results, are qualitatively consistent. See $ and SE in the appendix for more details.
5.1 TRIPLET ANNOTATION
We recruit crowdworkers on Prolific to acquire visual similarity triplets. In each question, we show reference image on top and two candidate images below, and ask a 2-Alhernative--ore Seeenative-Force--hhoeee Forced-Choice (2AFC) question: which candidate image looks more similar to the reference image? screenshot of the interface can be found in the appendix. To generate triplets for annotation, we first sample the reference image from either the training, the validation, or the test set. Then for each reference image, sample two candidates from the training set. We sample the candidates only from the training set because in decision support, the selected examples should always come from the training set, and thus only need to validate and test triplet accuracies with candidates from the training set.
For BM we recruit 80 crowdworkers, each completing 50 questions, giving us 4000 triplets. For CXR recruit 100 crowdworkers, each answering 20 questions, yielding 2000 triplets. Our pilot study suggests that visual similarity judgment on chest X-rays is more mentally demanding task, so we decrease the number of questions for each CXR survey.
5.2 RESULTS ON BUTTERFLIES V.S. MOTHS
We recruit crowdworkers on Prolific to evaluate representations produced by our models by doing decision support tasks. We acquire examples with different example selection policies from HC
7

---

Published as a conference paper at ICLR 2023
0.9
0.9
0.7
0.7
0.5
HC
0.5
MLB
HC
neutral
neutral
pesuasiveperuasive
neutral pesuasiveperussie
(a) Butterfly vs. Moth.
(b) Pneumonia classification.
Figure 4: Decision support accuracy with human subject studies. Error bars show 95% confidence intervals. HC dominates MLE in both neutral and persuasive decision support.
and MLE. We choose the dimension and training triplets of the representation based on the models' classification accuracy, triplet accuracy, and decision support simulation results based on synthetic agents. See more details in the appendix. We do not include TML in human studies, because in practice, TML models cannot make predictions class labels, therefore are unable to distinguish and select in-class and out-of-class examples and thus cannot be used for decision support.
H2H comparison results show HC NI examples are slightly but significantly preferred over MLE NI examples according to human visual similarity. We recruit 30 Prolific workers to make H2H comparisons between HC NI examples and MLE NI examples over the entire test set. The mean preference for HC over MLE is 0.5316 with a 95% confidence interval of +0.0302 (p = 0.0413 closer the test images than MLE NI examples with statistical significance according to human visual similarity.
with one-sample t-test). This means the HC NI examples
Decision support results show HC is significantly better than MLE both in neutral and persuasive decision support. Combining two example selection policies with two representations, we have four conditions: neutral, HC persuasive, MLE neutral, MLE persuasive. We also add a baseline condition with random supporting examples, which we call random in-class random out-of-class (RIRO). We recruit 30 Prolific workers for each condition and ask them go through the images in the test set with supporting examples from each class in the training set. Both the order of the test images and the order of the supporting images within each test question are randomly shuffled.
Figure 4a shows the human classification accuracies with different decision support scenarios and neutral decision support, we observe that HC achieves much higher accuracy than MLE (95.3% vs. 60.5%, p 4e- 19 with two-sample t-test). In fact, even RIRO provides better decision support than MLE representations, suggesting that the supporting images based MLE are confusing and hurt human decision making (77.5% vs. 60.5%, p 3)-6). As expected, the accuracies are generally higher in persuasive decision support. HC enables an accuracy of 97.8%, which is much better than MLE 79.5% (p 2e- 13). HC in neutral decision support already outperforms MLE in persuasive decision support. These findings confirm our results with VW synthetic experiments that human-compatible representations provide much better decision support than MIE representations.
different representations.
5.3 RESULTS ON CHEST X-RAYS
We use the same experimental setup as BM to evaluate HC and MLE representations in CXR.
H2H comparison results show HC NI examples are slightly preferred over MLE NI examples but the difference is not statistically significant. We recruit 50 Prolific workers each make 20 H2H comparisons between HC NI examples and MLE. NI examples. The mean preference for HC over MLE is 0,516 with 95% confidence interval of +-00725 (p = 0.379 with one-sample t-test). H2H comparison in CXR is especially challenging laypeople need to differentiate between chest X-rays in the same class, hence the slightly worse performance in H2H compared to B.M
Similar to BM, HC outperforms MLE in both neutral and persuasive decision support in CXR. As expected, Fig. shows that pneumonia classification is a much harder task than butterfly vs. moth classification, indicated by the lower accuracies across all conditions. In neutral decision support, HC enables much better accuracy than MLE (79.1% 1% vs. 63.8%, = 2 with two-sample t-test). In fact, similar the BM setting, MLE provides similar performance with RIRO (63.8% vs. 65.9%.
8

---

Published as a conference paper at ICLR 2023
p 0.390), suggesting that MLE representations selecting nearest neighbors within a class. To contextualize our results, we would like to highlight that our crowdworkers are laypeople and have no medical training. It is thus impressive that human- compatible representations enable an accuracy of almost 80% in neutral decision support, which demonstrates the potential of human-compatible representations.
different from random representations for
In persuasive decision support, HC provides the highest decision support accuracy at 90.0%, also much higher than MLE at 77. 77.0% (p- 10). Again, while do not recommend persuasive decision support as a policy for decision support in practice, these results show that our human-compatible representations are indeed more compatible with humans than MLE representations.
RELATED WORK
Ordinal embedding. The ordinal embedding problem (Ghosh et al., 2019 Van Der Maaten & Weinberger 20122 Kleindesseer & von Luxburg 2017, Kleindessner & Luxburg 2014] Terada & Luxburg 2014] Park et al. 2015) seeks to find low-dimensional representations that respect ordinal feedback. Currently, there exist several techniques for learning ordinal embeddings. Generalized Non- metric Multidimensional Scaling (Agarwal et al., (007) takes a max- max-margin approach by minimizing hinge loss. Stochastic Triplet Embedding (Van Der Maaten & Weinberger 2012) assumes the Bradley-Terry-Luce noise model (Bradley & Terry (9522]Leeee 1959) and minimizes logistic loss. The Crowd Kernel (Tamuz et al. 2011) and t-STE Van Der Maaten & Weinberger 2012) propose alternative non-convex loss measures based on probabilistic generative models. These results are primarily empirical and focus on minimizing prediction error unobserved triplets. In principle, one can plugin these approaches in our framework as alternatives to the triplet margin loss in Eq. [
AI explanations and Al-asiisted decision making. Various explanation methods have been devel- oped to explain black-box Al models (Guidotti et al. 2018), such as feature importance (Ribeiro et al.) 2016 Shrikumar et al. 2017), saliency map (Zhou al., 2016; Selvaraju al.. 2017), ), and decision rules (Ribeiro et al.] 2018). Example-based explanations are also a type of common explanation methods that use examples to explain AI models. Wearest-neightt
examples can explain a model's
local decision (Wang & Yin (202T] Nguyen et al. [021 Taesiri et al.) 2022] Lai & Tan 2019). To
the best of our knowledge, there has been no prior work that examines the role representations in choosing the nearest neighbors in the context of Al explanations. Meanwhile, global example-based
explanations such as prototypes can explain a model's global behavior or a model's understanding of the data distribution (Kim et al.. 2016 Chen et al. 2018a 8a Cai et al. 2019a] Lai et al.. 2020).
Explaining a model's 's global behavior is also closely related to machine teaching (Zhu et al. 2018).
Many of these explanation methods have been used in Al-assisted decision making to explain AI predictions or inform users about the AI model or training data (Lai et al.] (02II). Among
them, example-based explanations have shown be useful in many high-stake domains where full Al
automation is often not desired, such as recidivism prediction (Hayashi & Wakabayashi (017) and
medical diagnosis (Cai al. 2019b Rajpurkar et al.) 2020; Tschandl et al. | (020). While many of
the current literature in Al-assisted decision making focus on generating explanations of AI without
considering human feedback, our decision support methods offer assistance by learning from human perceptions and provide examples from human- compatbll representations.
7 CONCLUSION
Our work formulates the novel problem of learning huwan-compattin compatible representations for case- case-based decision support. As we identify in this paper, the key to providing effective case-based support with model is the alignment between the model and the human in terms of similarity metrics: two examples that appear similar to the model should also appear similar to the human. But models trained to perform classification do not automatically produce representations that satisfy this property. To address this issue, we propose a multi-task learning method to combine two sources of supervision: labeled examples for classification and triplets of human similarity judgments. With synthetic experiments and user studies, we validate that human-cou commannco representations (i) consistently get the best of both worlds in classification accuracy and triplet accuracy, (ii) select visually more similar examples in head-to head comparisons, (iii) and provide better decision support.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
We thank the anonymous reviewers for their insightful comments. We also thank members of the Chicago Human+AI AI lab for their thoughtful feedback. This work was supported in part by CDAC discovery grant at the University of Chicago and an NSF grant, IIS IIS-2049989.
ETHICS STATEMENT
Although coming from a genuine goal to improve human-Al AI collaboration by aligning Al models with human intuition, our work may have potential negative impacts for the society, We discuss these negative impacts from two perspectives: the multi-task learning framework and the decision support policies.
MULTI- TASK LEARNING FRAMEWORK
Our human-compatible representations models are trained with two sources of data. The first source of data is classification annotations where groundtruth maybe be derived from scientific evidence or crowdsourcing with objective rules or guidelines. The second source of data human judgment annotations where groundtruth is probably always acquired from crowdworkers with subjective perceptions. When our data is determined with subjective perceptions, the model that learns from may inevitably develop bias based on the sampled population. If not carefully designed, the human judgment dataset may contain bias against certain minority group depending on the domain and the task of the dataset. For example, similarity judgment based chest X-rays of patients in one gender group or racial group may affect the generalizability of the representations learned from it, and may lead to fairness problems in downstream tasks. It is important for researchers to audit the data collection process and make efforts to avoid such potential problems.
DECISION SUPPORT POLICIES
Among a wide variety of example selection policies, our policies to choose the decision support examples are only attempts leveraging AI model representations increase human performance. We believe that they are reasonable strategies for evaluating representations learned by model, but future work required to establish their use in practice.
The neutral decision support policy aims to select the nearest examples in each class, therefore limiting the decision problem to a small region around the test example. We hope this policy allow human users to zoom in the local neighborhood and scrutinize the difference between the relatively close examples. In other words, neutral decision support help human users develop a local decision boundary with the smallest possible margin. This could be useful for confusing test cases that usually require careful examinations. However, the neutral decision support policy adopts an intervention present small region in the dataset and may downplay the importance of global distribution in human users' decision making process.
The persuasive decision support policy aims to select the nearest in-class examples but the furthest out-of-class examples. It aims to maximize the visual difference between examples in opposite class, thus require less effort for human users adopt case-based reasoning for classification. It also helps human users to develop a local decision boundary with the largest possible margin. However, when model prediction is incorrect, the policy end up selecting the furthest in-class examples with the nearest out-of-class examples, completely contrary to what it is design to do, may lead to even over-reliance or even adversarial supports.
In general, decision support policies aim to choose number of supporting examples without considering some global properties such as representativeness and diversity. While aiming to reduce humans' effort required in task by encouraging them to make decision in local region, the decision support examples do not serve as representative view of the whole dataset, and may bias human users to have distorted impression of the data distribution. It remains an open question that how to ameliorate these negative influence when designing decision support interactions with case- based reasoning.
10

---

Published as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
Implementation details and computing resources are documented in $B in the appendix. Hyperpa- rameters and model configuration are reported in both the main paper and the appendix along with each experiments. Our code and data are available at https: /tlttp:://ithbbs /ggtttb..h...ggggggg com/ wwt. coom/ChicagoHAI/
lea Learning human coctanncccpatiiiil ible repressttttessssnnoooon
REFERENCES
Sameer Agarwal, Josh Wills, Lawrence Cayton, Gert Lanckriet, David Kriegman, and Serge Belongie. Generalized non-men non-metric multidimensional scaling. Artificial Intellignnce and Statisties, pp. 11-18, 2007.
Vassileios Balntas, Edgar Riba, Daniel Ponsa, and Krystian Mikolajezyk. Learning local feature descriptors with triplets and shallow convolutional neural networks. In Bmve, volume 1, pp. 3, 2016.
Shahina Begum, Mobyen Uddin Ahmed. Peter Funk, Ning Xiong, and Bo Von Schéelc. A case- based decision support system for individual stress diagnosis using fuzzy similarity matching. Computational Intelligence,
2009.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324 345, 1952.
Carrie J Cai, Jonas Jongejan, and Jess Holbrook. The effects of example-based explanations in machine learning interface. In Proceedings of the 24th international conference on intelligent user interfaces, pp. 258 268, 2019a
Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. "hello ai": Uncovering the onboarding needs of medical practitioners for human-ai collaborative decision- making. Proc. ACM 11t.1/....11532000000000
Interact. 3(CSCW), nov 2019b. doi: 10. 10.1145/3359206. URL
https:://di..oo
Chaofan Chen, Oscar Li, Alina Barnett, Jonathan Su, and Cynthia Rudin. This looks like that: deep learning for interpretable image recognition. CoRR, abs/1806. 10574, 2018a. URL http
//axxiv.crc/ab//IBB. 10574
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Yuxin Chen, Oisin Mae Aodha, Shihan Su, Pietro Perona, and Yisong Yue. Near-optimal machine teaching via explanatory teaching sets. In International Conference on Artificial Intelligence and Statistics, pp. 1970-1978. PMLR, 2018b.
Finale Doshi Velez and Been Kim. Towards rigorous science of interpretable machine learning. arXiv preprint arXiv: 077700000060 2017.
Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Representations, 2021. URL https::/openrev// net / Eerum id- YicTFaNTTy
William Falcon al. Pytorch lightning, 2019.
Nikhil Ghosh, Yuxin Chen, and Yisong Yue. Landmark ordinal embedding. In Advances in Neural Information Processing Systems, pp. 11502-11511 2019.
Ben Green and Yiling Chen. The principles and limits of algorithm in loop decision making. Proceedings of the ACM Human- Humon-Computer Interaction, 3(CSCW):1-24 2019.
11

---

Published as a conference paper at ICLR 2023
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):1-42, 2018.
Yugo Hayashi and Kosuke Wakabayashi. Can ai become reliable source support human decision making in court scene? In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW '17 Companion, pp. 195-198, New York, NY, USA, 2017. Association for Computing Machinery. doi: 10. R0.1144/02229883306666
URL
Satpss//doo. wttp:///ddoo.oo/////oooo 1145/ 332
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
Andrew Hyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019.
Daniel S. Kermany, Michael Goldbaum, Wenjia Cai, Carolina C.S. Valentim, Huiying Liang, Sally L. Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, Justin Dong, Made K. Prasadha, Jacqueline Pei, Magdalene Y.L. Ting, Jie Zhu, Christina Li, Sierra Hewett, Jason Dong, Ian Ziyar, Alexander Shi, Runze Zhang. Lianghong Zheng, Rui Hou, William Shi, Xin Fu, Yaou Duan, Viet A.N. Huu, Cindy Wen, Edward D. Zhang, Charlotte I.. Zhang, Oulan Li, Xiaobo Wang, Michael A. Singer, Xiaodong Sun, Jie Xu, Ali Tafreshi, M. An- thony Lewis, Huimin Xia, and Kang Zhang. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, 17265):1122-1131.cc 2018. ISSN 0092-8674. doi: http://t...rr//..........c..222.
WtRl:ttps:///mt http://wwwwweeeeeeeettt...t com/ sctiecce/artiilee/////883333333 448301545
Been Kim, Rajiv Khanna, and Oluwasanmi Koyejo. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems, 29, 2016.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Matthäus Kleindessner and Ulrike Luxburg. Uniqueness of ordinal embedding. In Conference Learning Theory, pp. 40-67, 2014.
Matthäus Kleindessner and Ulrike von Luxburg. Kerel functions based on triplet comparisons. In Advances in Neural Information Processing Systems, pp. 6807-6817, 2017.
Janet L Kolodneer. Improving human decision making through case-based decision aiding. Al magazine, 12(2):52-52, 1991.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. Advances in neural information processing systems, 25, 2012.
Vivian Lai and Chenhao Tan. On human predictions with explanations and predictions of machine learning models: case study on deception detection. In Proceedings of FAT* 2019.
Vivian Lai, Han Liu, and Chenhao Tan. "why 'chicago' deceptive?" towards building model-driven tutorials for humans. In Proceedings of CHI, 2020.
Vivian Lai, Chacha Chen, Q Vera Liao, Alison Emith-Renner, and Chenhao Tan. Towards science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv: 1471, 2021.
Shu-hsien Liao. Case- Case-based decision support system: Architecture for simulating military command and control. European Journal of Operational Research, 123(3):558-56) 2000.
12

---

Published as a conference paper at ICLR 2023
R Duncan Luce. Individual choice behavior. 1959.
C Metz, C Metz, N Tiku, Lapowsky, K Finley, C Thompson, E Griffith, and M Spector. In two moves, alphago and lee sedol redefined the future. wired, 2016.
Giang Nguyen, Daeyoung Kim, and Anh Nguyen. The effectiveness of feature attribution methods and its correlation with automatic evaluation scores. Advances in Neural Information Processing Systems, 34:26422-26436. 2021.
Dohyung Park, Joe Neeman, Jin Zhang, Sujay Sanghavi, and Inderjit Dhillon. Preference completion: Large-scale collaborative ranking from pairwise comparisons. In International Conference on Machine Learning, pp. 1907-1916, 2015. URL (RtLattp:///// /bitt 1.ttt.yy/20bbLL...
Adam Paszke, Sam Gross, Francisco Massa, Adam Lever, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf. Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019. URL http::://pppess.eeuziiiiee wwtpeurrrr.i.ccpppppeepeee 901 5 pytorch-ar mperat ve sty h per learning Libraryy
pdf
Pranav Rajpurkar, Chloe O'Connell, Amit Schechter, Nishit Asnani, Jason Li, Amirhossein Kiani,
Robyn L. Ball, Mare Mendelson, Gary Maartens, Daniël J. van Hoving, Rulan Griesel, Andrew
Ng. Tom H. Boyles, and Matthew P. Lungren. CheXaid: deep learning assistance for physician diagnosis tuberculosis using chest x-rays in patients with HIV. пpj Digital Medicine, 3(1):
1-8, September 2020. ISSN 2398-6352.
doi: 10.10038644744--0000003222
URL https
// www www.ua.tue...... www.attreecooeeeeeeeeeoe art wtt.//aaicccssss/ee
020 00322-2
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference computer vision, pp. 6186626, 2017.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, pp. 3145- 3153. PMLR, 2017.
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre,, George Van Den Driessche, Julian Schrittieeser, loannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484 489, 2016.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 350(7676):354-3555 2017.
Adish Singla, Ilija Bogunovic, Gábor Bartók, Amin Karbasi, and Andreas Krause, Near-optimally teaching the crowd to classify, In International Conference Machine Learning, pp. 154-162. PMLR, 2014.
Mohammad Reza Taesiri, Giang Nguyen, and Anh Nguyen. Visual correspondence--assee expla- nations improve ai robustness and human-al team accuracy. In Advances in Neural Information Processing Systems, 2022.
13

---

Published as a conference paper at ICLR 2023
Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, and Adam Tauman Kalai. Adaptively learning the crowd kernel. In Proceedings of the 28th International Conference on International Conference Machine Learning, pp. 673-680, 2011.
Yoshikazu Terada and Ulrike Luxburg. Local ordinal embedding. In International Conference on Machine Learning, pp. 847-855, 2014.
Philipp Tschandl, Christoph Rinner, Zoe Apalla, Giuseppe Argenziano, Noel Codella, Allan Halpem, Monika Janda, Aimilios Lallas, Caterina Longo, Josep Malvehy, John Paoli, Susana Puig, Cliff Rosendahl, H. Peter Soyer, Iris Zalaudek, and Harald Kittlr. Human-computer collaboration for skin cancer recognition. Nature Medicine, 26(8):1229-1234 August 2020. ISSN 1546- 170X. doi: 19911//////1111000099009 0215591-2000942--0
URLhttps: www. nature com www. artieest/
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
Laurens Van Der Maaten and Kilian Weinberger. Stochastic triplet embedding. In Machine Learning for Signal Processing (MLSP). 2012 IEEE International Workshop on, pp. 1-6, 2012. URL
https: 1y/.22TF/222223333
Xinru Wang and Ming Yin. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference Intelligent User Interfaces. IUI '1,, pp. 318-328, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450381171, doi: 10.1145/339748 34450600. URL (tLhhtts: org/ 3450650
Kai Xiao, Logan Engstrom, Andrew llyas, and Aleksander Madry. Noise signal: The role of image backgrounds in object recognition. arXiv preprint arXiv:2006. 09994, 2009990006009994 2020.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unrcasonable effectiveness of deep features perceptual metric. In CVPR, 2018.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921-2929, 2016.
Xiaojin Zhu, Adish Singla, Sandra Zilles, and Anna N Rafferty. An overview of machine teaching. arXiv preprint arXiv: /801.05977 2018.
14

---

Published as a conference paper at ICLR 2023
A LIMITATIONS
We discuss some of the limitations in our work.
Limitations of decision support policies. Our decision support policies are simple first steps towards more general example selection policy for decision support. There are certain limitations of our selection policies. For example in this work, we only look at selecting two examples from the two classes in binary image classifcation tasks. We encourage future work explore more selection methods towards effective decision-support.
In addition the ethical concerns discussed in the main paper and the ethics statement, neutral decision support and persuasive decision support policies have different limitations and use cases, Neutral decision support selects the nearest example from each class. Therefore when a test example lies too close to the decision boundary, the test example, in-class example, and out-of-class example may appear too similar to be distinguished by humans. This is where we may need select examples further away with different features so that users are more likely to spot the distinction. Persuasive decision support selects the most similar example in the predicted class and the least similar example in the other class, the latter of which has a risk of being an outlier. This may invite biases about the data distribution of the other class and degrade effectiveness of decision support.
Limitations of experimenting with crowdworkers. There are several limitations of experimenting with crowdworkers. First, crowdworkers may not invest as much time as domain experts in the tasks. Therefore, collected triplets may come from superficial or the salient features among the images. Second, crowdworkers or in general lay people have limited domain knowledge such as basic anatomy of body parts when working with medical image. Therefore it is less likely for them notice the most important feature in the images. In our CXR task, we mitigate this limitation by providing an instruction and quiz section before our main study that provides basic information about how to examine chest X-rays. However, in other tasks, we may need to provide more detailed instructions and quizzes help crowdworkers understand the task and this way polish collected triplets.
As the expertise level of the end users increases, HC should be able to learn a high-quality represen- tation. The effectiveness of our decision support methods may vary due to experts strong domain knowledge, but we would still expect our human-compaten decision support than MLE representations.
representation to provide more effective
Our ultimate goal is to apply our method to domain experts. We start with crowdworkers and the positive results are encouraging. We hope these results could be used to convince and invite more domain experts to get involved and work towards an applicable system together in the future.
Limitations of design choices in the algorithm. A number of decision choices were made in the algorithm. For example, we use Euclidean distance the distance metric to be learned for the representation space. Experimenting with different kinds of metrics (c.g., in the psychology literature) and exploring the effectiveness of their respective representations in decision support would be an interesting future direction.
We used ResNet the backbone network for feature extraction of images due to its competitiveness and popularity. Although model architecture is not the main concern of this paper, one could also plug in other common backbones such DenseNet (Huang et al. 2017) and ViT (Dosovitskiy et al.) 2021) into our representation learning algorithm. We leave the exploration of additional architecture and the effectiveness of their learned representation on decision support to future work.
B IMPLEMENTATION DETAIL
The architecture of our model is presented in Fig. 5 We first encode image inputs using a Convo- lutional Neural Network (CNN), and then project the output into an high-dimension representation space with projection head made of multi-layer perceptron (MLP). In our experiments we use one non-linear layer to project the output of the CNN into our representation space. For classifcation task we add an MLP classifier head. We also prediction, we re-index the representations with the current triplet batch and calculate prediction or loss. We use the PyTooch framework (Paszke al. 2019) and the PyTorch Lightning framework
mon-lineat layer with softmax activation. For triplet
15

---

Published as a conference paper at ICLR 2023
Embeddings
Classifier (MLP)
Class Probability
CNN (ResNet)
Projection (MLP)
Images (Batch)
Triplet Prediction
Figure 5: Architecture of the human-compatibl representations model.
Table 3: Classification and triplet accuracy of human- comaanco representations with different
TMLModel 1 has classfication head and classification accuray.
Model
Classification accuracy
Triplet accuracy
MLE
0.998 0.003
0.673 + 0.014
HC À 0.8
0.998 + 0.032
0.970 + 0.024
HC 0.5
0.995 0.000
0.972 + 0.004
HC À 0.2
0.996 + 0.016
0.973 + 0.039
TML
N/A
0.973 + 0.016
(Falcon et al., 2019) for implementation. Hyperparameters will be reported in CC for models in the synthetic experiments and in and {E for models in the human experiments.
B. COMPUTATION RESOURCES
We use computing cluster at our institution. We train our models on nodes with different GPUs including Nvidia GeForce RTX2080Ti. Nvidia GeForce RTX3090, Nvidia Quadro RTX 8000, and Nvidia A40. All models are trained one allocated node with one GPU access.
C SYNTHETIC EXPERIMENT RESULTS
C.I HYPERPARAMETERS
For our MLE backbone we use We use different controlling strength between classification and human judgment prediction, including As at 0.2, 0.5, and 0.8, and discuss the effect of À in the next section. In contrast to the experiments on BM, we observe that human-compatible representations with 51 512-dimension embedding shows overall better performance than human-compation with 50-dimension embedding and show results for the latter in the next section. We use the Adam optimizer (Kingma & Ba, 2014) with learning rate 1e 4. We use training batch size of 40 for triplet prediction, and 30 for classification.
representations
C.2 ADDITIONAL RESULTS
Classification and triplet accuracy. Table 3 how tuning À affects human-compatible repre- sentations's classification and triplet accuracy. Higher À drives human-coopatible representations to behave more simlar to MIE representations while lower human-compatible representations is more similar TMLModel.
Experiment results on VW with confidence intervals. Table presents results on VW with human-compatible representations 0.5. This is is simply Table in the main paper with 0.95 confidence intervals.
Results for different In Table and T we show experiment results with human- comaa-co representations using À = 0.2 and x 0.8. We do not observe clear trend between and evaluation metric performances. In the main paper we present human-come comaniblm representations with À 0.5 it shows best overall performance.
16

---

Published as a conference paper at ICLR 2023
Table 4: Experiment results on VW. Models use 512-dimension embeddings; HC uses À = 0.5 and filtered triplets. This the same table as Table and adds confidence intervals.
Alignments
50%
80%
83%
92%
92.5%
100%
Weights
[0.0.1.1]
[1.0.1.]]
[0.1.1.]]
[1.256,256.25]
(256,1,256,256]
[1.1.11]]
NI-H2H
MTB
0.917 0.064
0.914 + 0.007
0.903
0.016
0.880 + 0.022
0.872 + 0.020
0.808
0.017
NO-H2H
BC MLE
0.916 0.093
0,968 1 0.011
0.946 0.009
0.958 + 0.031
0.962 + 0.008
0.970 + 0.008
Neutral decision support
MLE
0.753 + 0.056
0.899 + 0.025
0.896 +0.044
0.897 + 0.045
0.901 + 0.025
0.929 = 0.028
TKL
0.568 1: 0.049
0.775 1 0.084
0.807 + 0.038
0.868 + 0.012
0.877 1: 0,025
1.000 - 0.000
RC
0.759 + 0.080
0.901 + 0.016
0.928 + 0.099
0.949 + 0.034
0.955 + 0.027
1.000 + 0.00
Persuasive decision support
MLE
0.704 0.028
0,900 1 0.017
0.903 10.017
0.903 1:: 0.017
0.901 +: 0.017
0.919 1 0.016
TVT,
0.906 0.011
0.881 + 0.043
0.863 0.044
0.876 + 0.027
0.877 + 0.076
1.000 + 0.000
BC
1.000 0.000
1.000 + 0.000
1.000 0.000
1.000 + 0.000
1.000 + 0.000
1.000 = 0.000
Table 5: Experiment results on VW. Models using 512-dimension embeddings; HC uses À = 0.2 and
filtered triplets.
Alignments 50%
80%
83%
92%
92.5%
100%
Weights
[0,0,1.1]
[1,0,1,1]
[0.1,1.1]
[1,256,256,256]
1,26,,,))
[1,1,1,]]
N1-H2H
FC M.E
0.920 +: 0.005
0.890 +: 0.032
0.906
0.053
0.895
0.016
0.862 + 0.254
0.832 + 0.058
NO-H2H
BC vs. MLE
0,901 = 0.439
0.948 + 0.095
0.970 + 0.019
0.972 1: 0.095
0.933 + 0.154
0.981 +: 0.040
Neutral decision support
MLE
0.753 1. 0.056
0.899 1 0.025
0.896 10.044
0.897 0.045
0.901 1 0.025
0.929 1 0.028
TMT.
0.568 + 0.049
0.775 + 0.084
0.807 0.038
0.868 0.012
0.877 0.025
1.000 + 0.000
BC
0.740 0.540
0.925 + 0.127
0.933 0.064
0,935 0.000
0.945 0.349
1.000 = 0.000
Persuasive decision support
0.704 0.028
0.900 + 0.017
0.903 +0.017
0.903 0.017
0.901 0.017
0.919 0.016
0.906 0.011
0.881 + 0.043
0.863 0.044
0.876 0.027
0.877 0.076
1.000 + 0.000
DC
0.996 + 0.016
0.995 + 0.000
0.998 0.000
0.996 0.016
0.995 0.000
0.995 = 0.032
Number of triplets. We examine the effect of the number of triplets, showing the results in Fig. We decrease number of triplets by powers of 2 and find that H2H preference towards human- compatible representations indeed declines as HC is less human-compatible with fewer training data. As for decision support, in neutral decision support HC performance declines and eventually approaches MLE representations except an outlier in the end, while in persuasive decision support HC performance is able stay 100% even as the number of triplets declines.
Additional details on weight generation. We generate aligmment scores by searching through weight combinations of the simulated human visual similarity metrics. We search the weights in powers of 2, from 0 to 210 producing sparse distribution of alignments (Fig. 7). Increasing search range to powers of 10 produces smoother distribution, but the weights are also more extreme and unrealistic. We note that the aligmment distribution may vary across different datasets. In our experiments we choose weights and alignments to be as representative to the distribution possible.
C.3 ADDITIONAL. DECISION BOUNDARIES
We create a variant of the VW dataset where the labels are populated by a linear separator. We refer to this dataset VW-Linear (Fig. g). We find the results overall similar to the original VW data.
17

---

Published as a conference paper at ICLR 2023
Table 6: Experiment results on VW. Models using 512-dimension embeddings; HC uses X = 0.8 and filtered triplets.
Alignments
50%
80%
83%
92%
92.5%
100%
Weights
[0.0.1.1]
[1.0.1.]]
[0.1.1.1]
[1.256,256.25]
(256,1,256,256]
[1.1.1.]]
NI-H2H
MTB
0.916 0.082
0.869 + 0.217
0.891
0.029
0.879 + 0.066
0.853
0.164
0.828
0.138
NO-H2H
MLE
0.902 0.193
0,944 1 0.093
0.959
0.005
0.956 + 0.090
0.942 + 0.026
0.969 + 0.034
Neutral decision support
MLE
0.753 + 0.056
0.899 + 0.025
0.896 +0.044
0.897 = 0.045
0.901 + 0.025
0.929 = 0.028
TKL
0.568 1: 0.049
0.775 1 0.084
0.807 + 0.038
0.868 + 0.012
0.877 1: 0.025
1.000 + 0.000
RC
0.740 + 0.095
0.894 + 0.111
0.929 + 0.079
0.960 + 0.032
0.923 + 0.127
1.000 + 0.000
Persuasive decision support
MLE
0.704 0.028
0,900 1 0.017
0.903 10.017
0.903 1:: 0.017
0.901 +: 0.017
0.919 1 0.016
TVT,
0.906 0.011
0.881 + 0.043
0.863 0.044
0.876 + 0.027
0.877 + 0.076
1.000 + 0.000
BC
0.998 0.032
+ 0.000
0.998 0.000
0.998 + 0.032
+ 0.000
0.999 = 0.016
Table 7: Experiment results on VW. Models use 512-dimension embeddings; HC uses = 0.5 and
unfiltered triplets.
Alignments 50%
80%
83%
92%
92.5%
100%
Weights
[0,0,1.1]
[1,0,1,1]
[0.1,1.1]
[1,256,256,256]
1,56,1,256,2561
[1,1111]]
N1-H2H
FC M.E
0.921 + 0.015
0.900 +: 0.035
0.920
0.023
0.895
0.008
0.867
0.034
0.846 + 0.016
NO-H2H
BC vs. MLE
0,951 = 0.034
0.969 + 0.024
0.991 + 0.002
0.991 0.004
0.958 +: 0.010
0.980 + 0.023
Neutral decision support
MLE
0.753 1. 0.056
0.899 1 0.025
0.896 0..044
0.897 0.045
0.901 1 0.025
0.929 1 0.028
TMT.
0.568 + 0.049
0.775 + 0.084
0.807 0.038
0.868 0.012
0.877 0.025
1.000 + 0.000
BC
0.603 0.051
0.801 + 0.025
0.848 0.053
0,880 0.000
0.880 0.081
1.000 = 0.000
Persuasive decision support
0.704 0.028
0.900 + 0.017
0.903 +0.017
0.903 0.017
0.901 0.017
0.919 0.016
0.906 + 0.011
0.881 + 0.043
0.863 0.044
0.876 0.027
0.877 0.076
1.000 + 0.000
DC
0.996 + 0.004
0.999 + 0.004
0.996 0.004
0.996 0.004
0.996 0.004
0.997 = 0.004
Classification and triplet accuracy. Table shows classification and triplet accuracy of tuning showing a similar trend to the previous experiment.
H2H and decision support results In Table 10 we present results with the best set of hyperparam- eter: filtered triplets, 112-dimension embedding, À = 0.5. We show results for À = 0.2 Table
and À = 0.8 in Table
Similar to the experiment on VW square decision boundary, we see no clear relation between. X, embedding dimension and our evaluation metrics.
D HUMAN SUBJECT STUDY ON BUTTERFLIES v.S. MOTHS
D.1 DATASET
Our BM dataset include four species of butterflies and moths including: Peacock Butterfly, Ringlet Butterfly, Caterpiller Moth, and Tiger Moth. An example of each species is shown in Fig
18

---

Published as a conference paper at ICLR 2023
Persuasive DS Acc.
H2H with MLE
Neutral DS Acc.
1.0
1.0
1.0
0.8
0.9
0.6
0.8
0.9
0.4-
0.2
0.7
IIC
0.8
IC
0.0
0.6
20089
10080
sae
2500
1250
623
20009
10000
52l0
2500
20000
10000
5000
2300
1,50
Number of triplets
Number of triplets
Number of triplets
Figure 6: HC performance declines the number of triplets decreases, but shows strong persuasive decision support accuracy even with very few triplets.
6000
4000
2000
Align (%)
Figure 7: Histogram of alignments generated by searching informative weights in powers of 2.
D.2 HYPERPARAMETERS
We use different controlling strength between classification and human judgment prediction, including As at 0.2, 0.5, and 0.8. We use the Adam optimizer (Kingma & Bal 20144) with learning rate 1e Our training batch size is 120 for triplet prediction, and 30 for classification. All models trained for 50 epoches. The checkpoint with the lowest validation total loss in each run is selected for evaluations and applications.
D.3 CLASSIFICATION AND TRIPLET ACCURACY
We present the test-time classification and triplet accuracy of our models in Table 13 Both MLE and HC achieve above 97.5% classification accuracy. HC in the 512-dimension unfiltered setting achieve 100.0% classification accuracy. Both TML and HC achieve above 70.7% triplet accuracy. Both TML and HC achieve the highest triplet accuracy in the 50-dimension unfiltered setting with triplet accuracy 75.9% and 76.2% respectively. Filtering class-inconsistent triplets removes 15.75% of the triplet annotations this dataset.
We also evaluate the pretrained LPIPS metric (Zhang al. 2018) for learning perceptual similarity. Results with AlexNet backbone and VGG backbone are at 54.5% and 55.0% triplet accuracy respectively, suggesting that TML and HC provides much better triplet accuracy in this task.
triplet test baselines
D.4 EFFECT OF TRIPLET AMOUNT AND TYPE
We evaluate the effect of the number of triplets on our models in Fig. 10 Similar to the VW experiments, H2H preference towards human-compatiol representations and neutral decision support performance decrease as the number triplets decreases. Human-compatiln representations achieve strong persuasive decision support performance with very few triplets.
19

---

Published as a conference paper at ICLR 2023
Table 9: HC performance with different À on VW linear decision boundary data.
Model
Classification accuracy
Triplet accuracy
MLE
0.993 1 0.003
0.673 1 0.014
HC = 0.8
0.988 0.032
0.968 = 0.030
0.5
0.978 0.013
0.966 = 0.007
HC À 0.2
0.978 + 0.032
0.970 + 0.010
Figure 8: VW-Linear
TML
N/A
0.976 + 0.012
Table 10: Experiment results on VW-Linear. Models use 512-dimension embeddings; HC uses
0.5 and filtered triplets.
Alignments
56%
84%
95%
98.5%
Weights
[0.1,1.1]
[1,0.1,1]
[1,1,11]]
[32,256,111]
NI-H2H
EC vs. MLE
0.913 + 0.023
0.922 + 0.008
0.899 + 0.020
0.848 + 0.055
NO-H2H
EC vs. MLE
0.932 + 0.034
0.960 0.027
0.921 + 0.013
0.928 + 0.034
Neutral decision support
0.778 + 0.084
0.792 0.144
0.839 + 0.130
0.927 + 0.019
TMI,
0.554 + 0.175
0.770 0.318
0.950 + 0.095
0.914 + 0.075
HC
0.841 + 0.053
0.911 + 0.053
0.967 + 0.009
0.961 + 0.014
Persuasive decision support
MLE
0.802 + 0.249
0.815 0.151
0.848 + 0.188
0.953 - 0.051
TML
0.473 + 1.016
0.653 + 1.747
0.441 + 0.016
0.381 = 0.474
EC
0.979 1 0.014
0.977 1: 0.009
0.977 1 0.009
0.978 1 0.013
D.S MODEL EVALUATION WITH SYNTHETIC AGENT
We trained models with different configurations. We mainly discuss two factors: 1) filtering out inconsinnonitttnt triplets or not; 2) a large dimension at 512 vs. a small dimension at 50 for the output representations. We also tried different hyperparameters such as different Às that control the strength of the classification loss and triplet margin loss as well as different random seeds. We select the best TML HC / MLE in each filtring-dimensiio configuration with the highest average of test. classification accuracy and test triplet accuracies.
Label accuracy and triplet accuracy. As this task is relatively simple, both MLE and HC achieves test accuracy of above 97.5%. In fact, HC without filtering out class-inconsistentt triplets achieved 100%. Note that TML cannot classify alone. As for triplet accuracy, expected, both and TML outperform MLE. Dimensionality does not affect triplet accuracy, but filtering out inconsincnn triplets decrease triplet accuracy (76.2% vs. 70.7% with 50 dimensions, 74. 1 % vs. 70.9% with 512 dimensions). This is because filtering creates distribution shift of the triplet annotations, and limits the models' ability learn general human visual similarity.
To run synthetic experiments for case-based decision support, we select the TML with the best test triplet accuracy as our synthetic agent, and then evaluate the examples produced by representations. We do not show results of TML use it as the synthetic agent.
Human-compatible representations is prefered over MLL representations in H2H. We compare examples selected from different models in different configurations to examples selected by the MLE baseline with the same dimensionality.
Table 15 shows how often the synthetic agent prefers the tested model examples to baseline MLE examples. In all settings, the preference towards HC is above 50%, but not as high as those in our synthetic experiments with the VW dataset. Filtering out class-inconsister triplets improves the preference for the nearest example with the predicted label, while hurting the preference for the nearest out-of-class example.
20

---

Published as a conference paper at ICLR 2023
Table 11: Experiment results on VW-Linear. Models use 512-dimension embeddings; HC uses 0.2 and filtered triplets.
Alignments
56%
84%
95%
98.5%
Weights
[0.1,1,1]
[1,0,1,1]
[1,1,11]]
[32,256,1,1]
N1-H2H
HC vs. MLE
0.936 + 0.024
0.921 + 0.008
0.912 + 0.074
0.856 + 0.034
NO-H2H
EC vs. MLE
0.946 + 0.032
0.974 + 0.032
0.949 + 0.003
0.934 + 0.029
Neutral decision support
MLE
0.778 1 0.084
0.792 0.144
0.839 1 0.130
0.927 + 0.019
TML
0.554 1 0.5
0.770 + 0.318
0.950 1 0.095
0.914 1 0.075
EC
0.845 = 0.127
0.880 + 0.127
0.956 + 0.016
0.956 = 0.111
Persuasive decision support
MLE
0.802 + 0.249
0.815 + 0.151
0.848 +: 0.188
0.953 1 0.051
TML
0.473 = 1.016
0.653 1.747
0.441 + 0.016
0.381 = 0.474
EC
0.974 = 0.016
0.970 0.064
0.968 + 0.064
0.988 = 0.032
Table 12: Experiment results on VW-Linear. Models use 512-dimension embeddings; HC uses 0.8 and filtered triplets.
Alignments
56%
84%
95%
98.5%
Weights
10,1,1,11
[1,0,1,11
[1,1,111]
132,256,111]
NI-H2H
EC vs. MLE
0.906 + 0.122
0.909 + 0.111
0.882 + 0.135
0.848 + 0.050
NO-H2H
EC vs. MLE
0.926 = 0.021
0.955 0.199 0.936 + 0.053
0.912 = 0.095
Neutral decision support
0.778 0.084
0.792 0.144
0.839 + 0.130
0.927 0.019
TML
0.554 + 0.175
0.770 0.318
0.950 + 0.095
0.914 = 0.075
EC
0.824 = 0.175
0.895 + 0.159
0.950 + 0.032
0.969 = 0.016
Persuasive decision support
MLE
0.802 + 0.249
0,815 0.151
0.848 + 0.188
0.953 : 0.051
TML
0.473 + 1.016
0.653 + 1.747
0.441 i 0.016
0.381 : 0.474
HC
0.981 + 0.048
0.964 + 0.206
0.961 + 0.175
0.978 = 0.064
Decision support simulations shows large dimension benefits MLE representations but hurts unfiltered human-compatible representations in neutral decision support. We also run simulated decision support with the TML synthetic agent. Table shows decision support accuracy for different settings. MLE have both higher neutral decision support accuracy and persuasive decision support scores when we use a large dimension at 512. We hypothesize that for MLE, reducing dimension may force the network to discard dimensions useful for human judgments but keep dimensions useful for classification. We then use the 51 512-dimension MLE with the highest intrinsic evaluation scores as our MLE baseline in later studies.
For HC, neutral decision support accuracy are in general comparable to 87.5% score of the 512- dimension MIE baseline except unfiltered 51 512-dimension HC which has only 80%. We hypothesize that representations of large dimension may struggle more with contradicting signals between metric learning and supervised classification in the unfiltered settings. For persuasive decision support, achieves perfect scores in all settings.
Overall, to proceed with our human- subject experiments, we choose HC filtered with 50 dimensions as our best HC as it achieves good balance between H2H and neutral decision support. For MLE, we choose the representation with 512 dimensions, We conduct head-to-head comparison between these
21

---

Published as a conference paper at ICLR 2023
(a) Ringlet Butterfly
(b) Peacock Butterfly
(c) Caterpiller Moth
(d) Tiger Moth
Figure 9: An example each species the BM dataset.
Table 13: Classification and triplet accuracy of BM models.
Model
Classification accuracy
Triplet accuracy
Dimension 50
MLE
0.975
0.610
HC
0.975
0.762
0.975
0.707
TML
N/A
0.759
TML-filtered
N/A
0.721
Dimension 512
MLE
0.975
0.631
HC
1.000
0.741
HC-filtered
0.975
0.709
TML
N/A
0.748
TML-filtered
N/A
0.732
two representations. Our synthetic agent prefers HC in 70% of the nearest in-class examples and in 97.5% of the nearest out-of-class examples.
D.6 INTERFACE
We present the screenshots of our interface at the end the appendix. Our interface consists of four stages. Participants will see the consent page the beginning, shown in Fig 13 After consent page, participants will see task specific instructions, as shown Fig [5] After entering the task. partipants will see the questions, studies to check whether participants are paying attention to the questions. Following suggestions on Prolific, design the attention check with explicit instructions, as shown in Fig T8 After finishing all questions, participants will reach the end page and return to Prolific, as shown in Fig 20 Our study is reviewed by the Institutional Review Board (IRB) institution (IRB22-0388).
shown in Fig [6 We also include two attention check questions in all
D.7 CROWDSOURCING
We recruit our participants on a crowdsourcing platform: Prolific wwwwpprolific.co) [April-May 2022]. We conduct three total studies: an annotation study, decision support study, and head-to-head comparison study. We use the default standard sampling on Prolific for participant recruitment. Eligible participants are limited to those reside in United States. Participants are not allowed to attempt the same study more than once.
Triplet annotation study We recruit 90 participants in total. We conduct pilot study with 7 participants to test the interface, and recruit 83 participants for the actual collection of annotations. 3 participants fail the attention check questions and their responses excluded in the results. We spend in total $76.01 with an average pay at $10.63 per hour. The median time taken to complete the study is 3'22".
Decision support study We recruit 161 participants total. 3 participants fail the attention check questions and their responses excluded in the results. We take the first 30 responses in each
22

---

Published as a conference paper at ICLR 2023
Persuasive Acc.
H2H with MLE
Neutral DS Acc.
1.0
1.0
0.95
0.9
0.90
0.8
0.85
0.7
0.8
0.80
0.75
0.6
2000
0.7
300
Number of triplets
Number of triplets
Number of triplets
Figure 10: performance declines the number of triplets decreases, but shows strong persuasive decision support accuracy even with very few triplets.
Table 16: BM decision support accuracy with
Table BM H2H preference results with syn- thetic agent.
synthetic agent.
Dimensions
50
512
Dimensions
50
512
Neutral Decision Support
NI H2H with MLE
MLE 0.675
0.875
HC
0.838
0.575
HC 0.900
0.800
HC filtered
0.863
0.725
HC filtered
0.875
0.900
NO H2H with MLE
Persuasive Decision Support
HC
0.775
0.925
MLE
0.825
0.875
HC filtered
0.700
0.775
HC
1.000
1.000
EC filtered
1.000
1.000
conditon to compile the results. We spend in total $126.40 with an average pay at $9.32 per hour. The median time taken to complete the study is 3'53".
Head-to-head comparison study We recruit 31 participants in total, where participant fail the attention check questions and their responses excluded in the results. We spend in total $24.00 with an average pay at $9.40 per hour. The median time taken to complete the study is 3'43".
E HUMAN SUBJECT STUDY ON CHEST X-RAYS
E. DATASET
Our CXR dataset constructed from subset of the chest X-ray dataset used by |Krranyy al.) (018)), which had 5,232 images. We take a balanced subset of 3,166 images, 1,583 characterized as depicting pneumonia and 1,583 normal. The pneumonia class contains bacterial pneumonia and viral pneumoia images, but we do not differentiate them for this study. An example of each image class is shown in Fig
E.2 HYPERPARAMETERS
For CXR experiment, instead of ResNet-18 pretrained from ImageNet, use ResNet-18 finetuned on CXR classifcation as our CNN backbone, as we observe it provides better decision support simulation results. For training our HC model we use À of 0.5. We use the Adam optimizer (Kingma
& Ba 2014) with learning rate le 4. Our training batch size is 16 for triplet prediction, and 30 for classification. All models are trained for epoches. The checkpoint with the lowest validation total loss in each run is selected for evaluations and applications.
E.3 CLASSIFICATION AND TRIPLET ACCURACY
We present the test-time classification and triplet accuracy of our models in Table 17 Both MLE and HC achieve above 95% classification accuracy. Both TML and нс achieve above above 65% triplet accuracy. Both TML model and HC achieve the highest triplet accuracy in the 512-dimension unfiltered setting with triplet accuracy at 69.1% and 72.2% respectively. Filtering out class inconsistent triplets removes 20.69% of the triplet annotations in this dataset.
23

---

Published as a conference paper at ICLR 2023
(a) Normal
(b) Bacterial pneumonia
(c) Viral pneumonia
Figure 11: An example of each image class in the CXR dataset.
Table 17: Classification and triplet accuracy of CXR models.
Model
Classification accuracy
Triplet accuracy
Dimension 50
MLE
0.973
0.571
0.954
0.576
HC-filtered
0.955
0.574
TML
N/A
0.602
TML-fi feered
N/A
0.587
Dimension 512
MLE
0.973
0.588
HC
0.968
0.602
HC-filtered
0.971
0.561
TML
N/A
0.618
TML-filtered
N/A
0.591
E.4 MODEL EVALUATION WITH SYNTHETIC AGENT
Similar to the BM setting, we select the TML with the best test triplet accuracy as our synthetic agent, and then evaluate the examples produced by all representations. As table [8 shows, preference for HC over MLE H2H less significant compared to BM, likely due the challenging nature of the CXR dataset. We still observe the patten that filtering improves H2H performance.
Table 19 shows decision support accuracy for different settings. All models benefit from a large dimension at 512. We observe consistent patterns such filtering leading to better decision support.
E.5 EFFECT OF TRIPLET AMOUNT AND TYPE
We evaluate the effect of the number of triplets on our models in Fig. 12 Similar to the BM experiments, H2H preference towards human compatile representations and neutral decision support performance decrease as the number of triplets decreases. Human-compatible representations achieve strong persuasive decision support performance even with very few triplets.
E.6 INTERFACE
Our CXR interface mostly the same as our BM interface, except that we add basic chest X-ray instructions as participants may not be familiar with medical images. After the consent page at the beginning, participants will see basic chest X-ray instructions showing where the lungs and hearts. Then, they enter an multiple-choice attention check, as shown in Fig [4] The correct answer in "lungs and adjacent interfaces". Failing the attention check will disqualify the participant. After correctly answering the pre-task attention check, participants will see the same task specific instructions as in the BM studies, as shown in Fig [5] Screenshots of questions are shown in Fig 17 We also include two in-task attention check questions simlar to the B.M study. Our study is reviewed by the Institutional Review Board (IRB) at our institution with study number that we will release upon acceptance preserve anonymity.
24

---

Published as a conference paper at ICLR 2023
Table 19; CXR decision support accuracy with
Table 18: CXR H2H preference results with syn- thetic agent.
synthetic agent.
Dimensions
50
512
Dimensions
50
512
Neutral Decision Support
NT H2H with MLE
MLE
0.711
0.726
HC
0.536
0.675
HC
0.742
0.779
HC filtered
0,472
0.599
-illeeee
0.732
0.804
NO H2H with MLE
Persuasive Decision Support
HC
0.535
0.635
MLE
0.881
0.882
HC filtered
0.487
0.494
HC
0.949
0,966
HC-filtered
0.948
0.946
Persuasive DS Acc.
Neutral DS Acc.
H2H with MLE
1.0
1.0
RC
0.9
MLE
MLE
0.6
0.8
0.4
0.8
0.7
0.2
HC
0.6
0.7
0.0
1200
2400
1280
608
2000
660
Number of triplets
Number of triplets
Number of triplets
Figure 12: HC performance declines as the number of triplets decreases, but shows strong persuasive
decision support accuracy even with very few triplets.
E.7 CROWDSOURCING
We recruit our participants on Prolific (www.prolific.co) [September 2022]. We conduct three total studies: annotation study, decision support study, and a head-to-head comparison study. We use the default standard sampling on Prolific for participant recruitment. Eligible participants are limited those reside United States. Participants are allowed to attempt the same study more than once.
Triplet annotation study We recruit 123 participants in total. 20 partipants fail the pre-task attention check question and 3 participants fail the in-task attention check questions; their responses are excluded in the results. We spend in total $80.00 with an average pay S10.70 per hour. The median time taken to complete the study is 3'22".
Decision support study We recruit 296 participants in total. 34 partipants fail the pre-task attention check question and 10 participants fail the in-task attention check questions; their responses are excluded the results. We spend in total $221.67 with an average pay $11.00 per hour. The median time taken to complete the study is 3'"0".
Head-to-head comparison study We recruit 57 participants in total. partipants fail the pre-task attention check question and participants fail the in-task attention check questions; their responses are excluded in the results. We spend total $40.00 with an average pay $10.54 per hour. The median time taken to complete the study is 3'25".
25

---

Published as a conference paper at ICLR 2023
Welcome study:
Description:
judge
inoges may
recog
may
recogris object image. unower survey sporning stuty should about minutee. porticipation uchurtory.
study, We any parsonal seraliie cusstions riight
incendives: You be compersooned 1.00 completing the shady (aboux 5112 Dolir in the event of incompplenn work, you must contoot research leam permatation will determined ussed what compptodd and researchera discrtton.
TASE NOTE: This contrine attention checks suew that cipertts timnning
instructio
rejected
tmages
renomoppotoo may
lookking these images,
please participate study, Ofterwiie, your parto pution studly involie
bayond everycas
You max benefit from this stuty by leoming to reoog bird and bnsect spedess the iitss from this study help adwa poesible ways that humann interct with modilt, Your interuction with our Almodel may load to letter tran rig of diffirunt profession swen as nadisdighitt, and as reaull butter hcart.core. rcor.
Confidentialiy:: your Prediti Woree will used dicrtiuute th
the research collect
Mlenditable be
team.
identiledd musion study may used future research ataaeed with ciher researchrra for ruture ressarch without your addittornllaftooe conseet.
which
clinngg on button below, you that form, years and agroe particiiaate ressarch.
Figure 13: The consent form page on our interface.
Poucre
Phuse
(a) Basic instructions about chest X-rays.
(b) Multiple-choice attention check CXR tasks.
The correct answer is "lungs and adjacent interfaces".
Figure 14: Pre-task instructions and attentions check for CXR tasks
Welcomet thet barlyy dial siaiity
Metcore onlyy vieal Pheas
(a) The annotation and head to-head comparision task instructions.
(b) decision support instructions.
Figure 15: The task sascifp instruction page our interface.
26

---

Published as a conference paper at ICLR 2023
Gentinal 1142
Helencca
(a) The annotation and head-to-head comparision task questions.
(b) The decision support task questions.
Figure 16: The task-specific questions for BM.
nage!
(a) The annotation and head-to- to-head comparision task questions.
(b) The decision support task questions.
Figure 17: The task-specific questions for CXR.
Velertnce Ice
pabnt
Pawry
(a) The annotation and head-to-head comparision task (b) The decision support task attention check ques- attention check questions.
tions.
Figure 18: The task-specific attention check questions for BM.
27

---

Published as a conference paper at ICLR 2023
Thank you! Please answer the survey below.
1. Hos mary queations you 1iikk you havs answored correcly? fsccest for atannionn chee
Current
you
Strongly
Decageee
Dreagtee
Neutrall
Agree
Agree
Strengb
yourrvee general leedbeck staty? [Dptonna
Figure 19: The survey page of the decision support task interface.
Thun This of suredt nenobes
Thank you This and the sument senster
(a) annotation and head-to- seadd comparision task end page.
(b) The decision support page.
Figure 20: The task-specifer end page on our interface.
28


Reviews:
[{"review_id": "yMDgg_dIjNo", "reviewer": "Reviewer_7Qs2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors combine metric learning with classification to propose a data collection process and loss function for training models that are both accurate classifiers and learn representations that align with human perceptions of similarity in the data. The human-aligned representation can be used for case-based decision support, including justification, where a prediction is given and supported by similar training data with ground-truth labels, and neutral decision support, where no prediction is given, but the most similar labeled training data is given for each label to aid the human decision making process. Experiments on synthetic data with simulated humans show major improvement over a pure classifier for both justification and neutral decision support. Experiments on image classification with human subjects show minor or insignificant improvements in justification, but major and significant improvements in neutral decision support.  ", "summary_of_the_review": "With the disclaimer that I don't have any background knowledge in decision support methods, I think this is a good paper with a few small but important concerns and missing pieces, detailed in the \"Weaknesses\" part of my \"Strengths and Weaknesses\" section. If these can be addressed, I will recommend it for acceptance.", "strength_and_weaknesses": "Strengths\n\nThe presentation is clear and the paper was easy and enjoyable to read. The work addresses an important and challenging problem and shows major improvement over baselines. The discussion of the ethics of decision support strategies and the difference between \"neutral\" and \"persuasive\" models is appreciated, and I get the impression that the work is well thought out and potential concerns are addressed in good faith. The results using human subjects are appreciated and make the claims credible.\n\nWeaknesses (and questions and requests for clarification)\n\nThere is hardly any discussion of context and prior work in decision support. As a result, it is not clear to me how novel the methods are. Additionally, I'm not sure whether there is any previous work aimed at the particular decision support problems addressed in this work that would function as baselines for the proposed methods. If not, this should be explicitly stated, as the baselines used here are naive and fairly weak.\n\nThe purpose of the \"justification\" mode of decision support is not clear, and it should be better motivated. What is the goal? If the model's prediction is given, how would it improve human decision accuracy? Is it meant to earn trust? To be explainable? If the goal is related to trust or explainability, there seems to be an unstated assumption that training data with ground-truth labels can be blindly trusted and/or are themselves useful explanations, which seems insufficient. Whatever this goal is, is it reflected in the H2H score for evaluation?\n\nI have concerns about the neutrality of filtering of class-inconsistent triplets. With respect to neutral decision support, you state that \"the goal is not simply to maximize human decision accuracy, because one may use policies that intentionally show distant examples to nudge or manipulate human towards making a particular decision. Choosing nearest neighbors in each class is thus an attempt to present faithful and neutral evidence from the representation space so that humans can make their own decisions, hence preserving their agency. Therefore, the chosen nearest neighbors should be visually similar to the test instance by human perception [...]\". The acknowledgment of this nuance is appreciated and shows that the ethical aspect of decision support is well considered. However, it seems to me that class-inconsistent triplet filtering is itself a roundabout way to allow the model to be \"persuasive\" rather than \"neutral\" by learning a representation that is only human-aligned when convenient, but not human-aligned when it does not result in a compelling argument for the model's predicted label. This could explain the gains in performance by using filtering for neutral decision support. I think that this should at least be addressed in the paper; ideally, the human subject experiments should also include results without filtering. I understand that this may not be possible in the timeline for revisions, however. Regardless, the neutral decision support results are impressive even with this concern.\n\nThe H2H score does not take into account the accuracy of the model, so it's hard to say that a HC model is useful for justification based on H2H alone. I would at least like to see the classification accuracy of the HC and MLE models. It would be even better if there is some way to consider accuracy as part of the the H2H score itself. For instance, one way might be to define the H2H score as, considering only data where the MLE prediction is correct, the fraction of such data that HC is preferable *and makes the correct prediction* so that cases that are well-justified but incorrect are not counted in its favor.\n\nIt's not clear how to interpret the relationship between task alignment and decision support results in the synthetic experiments.\n\nIt's not clear why we see a large disparity between H2H results for synthetic and human subject experiments. This should be addressed. Maybe it makes a difference whether the same human(s) (or synthetic humans) are used for both training data collection and testing?\n\nMinor nit-picks:\n- End of section 3, under \"Head-to-head comparisons\", the sentence \"In addition to the typical justification for the predicted label, we also examine\nthat for the other class as those examples will be used in decision support\": I cannot tell what the latter half of this statement is trying to say.\n- Table 1, \"Persuasive decision support\", last column: the 1.000 in the middle row should be bold.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "s3mQWqHtr7o", "reviewer": "Reviewer_J1hq", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper the authors tackle the problem of case-based decision support. In the problem setting, a ML algorithm assists humans in making decisions on classification tasks by identifying an example from the training set that is similar to the unseen example. In the general case, the similarity between examples is determined by the metric space identified by the ML algorithm while training. However, this metric space can be very different from the space considered by the humans. This can cause the human judge to be misled. To tackle the issue, the authors propose a loss function which combines both MLE and a similarity loss. The similarity loss is derived from a triple learning problem (triplet margin loss). Results on both synthetic data and human experiments show that the proposed method performs much better than existing approaches.", "summary_of_the_review": "The authors tackle an important problem and provide an important solution. Through multiple experiments they show that the proposed method is indeed better. However, the proposed approach is not significantly novel. Furthermore, more experiments are required with humans who are field experts and problems that are not in the image domain. These aspects reduces the impact of this paper a bit.", "strength_and_weaknesses": "Strengths:\nThe authors tackle an important problem. The proposed approach is simple yet intuitive. Combining two different losses is the correct approach to tackle this issue. Moreover, the experiments are well designed and consider various issues that might pop luck (like data augmentation gain). Finally, the paper is well written and easy to follow.\n\nWeaknesses: \nIn cases where human judgement is necessary, often time the user would be an expert. Their judgment might vary a bit from the regular individuals. In such cases, the gain might be different from what is found in this paper. Furthermore, even though the paper is well written there are still some typos left. I would encourage the authors to fix those. The authors should also evaluate if the results hold for models other than ResNet. Furthermore, the authors can also consider how the approach can be used for other domains that is not image classification. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "UqPUW5DEfi4", "reviewer": "Reviewer_5Yxd", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focuses on the setting of case-based decision support where, in addition to receiving a model's prediction, a human decision-maker can also observe cases with similar-looking examples from the training set of the same label (justification), or provide similar looking examples with other labels (calibration). Specifically, the authors propose a loss function combining standard cross entropy loss with a triple margin loss that aligns representations closer together based on human judgements of similarity. ", "summary_of_the_review": "Overall, I think the paper addresses an interesting and relevant problem, and it's main weaknesses are novelty and clarity on certain experiment choices I mentioned in the review, and I'm happy to raise my score if the authors address them. ", "strength_and_weaknesses": "S1: The paper tackles an interesting and well-defined problem of learning more human-compatible representations in the specific context of case-based decision support. \n\nS2: The paper is very clearly written. \n\nS3: The proposed approach, HC, strongly outperforms baselines in two real-world user study experiments. \n\nW1: The details in 4.1 on the simulated human perceptual similarity metrics are very sparse and unclear, and it's not obvious whether this is a justifiable model of human perceptual judgements. \n\nW2: The filtering of class-inconsistent triples is a bit concerning, as ideally the proposed method would be able to leverage information captured by such \"noise\" in annotations. It would be interesting to see how much filtering was performed for the experiments. \n\nQ1: Could the authors clarify whether the model's predictions were shown to uses in the decision-support task and if not, why they were omitted, as case support is typically an augmented task (providing justification for a model's output)? It is not clear from the interface, and I'm not sure how relevant the results are if these outputs are omitted, as that would be a natural thing to include in a real world set-up.   ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "U5OS39JrAn", "reviewer": "Reviewer_BaWn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for jointly learning a case-based predictive algorithm that maximizes accuracy and alignment with human assessments of similarity. The paper provides a loss function for this purpose and describes in detail experiments on synthetic and real-world data annotated by human labelers on Prolific. The real-world datasets describe chest x-rays (CXR) and moth vs. butterfly image classification.  ", "summary_of_the_review": "This paper provides a simple solution to the well-posed problem: how to provide case-based decision support that aligns with human assessments of similarity while still being faithful to the predictions? The proposed method combines existing work to answer this, and the authors provide good empirical evaluations. The paper would be stronger if it provided more discussion on limitations and use cases.", "strength_and_weaknesses": "+ The paper considers an important and impactful problem: how to build decision support tools that are interpretable in a faithful way to the prediction. The motivation was really strong and well-communicated.\n+ The empirical evaluation is detailed and the results are interesting.\n\n- The discussion on limitations is lacking. What situations may it be insufficient to provide only the most similar examples in each class? The ethics discussion at the end begins to discuss these limitations, but I think it would be useful to elaborate on this and put it some discussion in the main paper since it has implications for effectiveness. It could be illustrative for instance to give two examples: one setting that is well-suited to this method and another that is not. \n- It would also be helpful to provide caveats about the empirical setup. For instance, what limitations result from having Prolific users (who are presumably not experts at reading x-rays) do the chest x-ray similarity assessments?\n\nPlease see next section for more on strengths and weaknesses with respect to novelty and clarity.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster