Original: A Uniﬁed Framework for Comparing Learning Algorithms
Revised: A Unified Framework for Comparing Learning Algorithms

Original: Anonymous Author(s)

Original: AfﬁliationAddress

Original: Abstract
Revised: Abstract

Original: Understanding model biases is crucial to understanding how models will performout-of-distribution (OOD).
Revised: Understanding model biases is crucial to understanding how models will perform out-of-distribution (OOD).

Original: These biases often stem from particular design choices(e.g., architecture or data augmentation).
Revised: These biases often stem from particular design choices (e.g., architecture or data augmentation).

Original: We propose a framework for (learning) algorithm comparisons , wherein the goal is to ﬁnd similarities and differences between models trained with two different learning algorithms.
Revised: We propose a framework for (learning) algorithm comparisons , wherein the goal is to find similarities and differences between models trained with two different learning algorithms.

Original: We beginby formalizing the goal of algorithm comparison as ﬁnding distinguishing feature transformations , input transformations that change the predictions of modelstrained with one learning algorithm but not the other.
Revised: We begin by formalizing the goal of algorithm comparison as finding distinguishing feature transformations , input transformations that change the predictions of models trained with one learning algorithm but not the other.

Original: We then present a two-stagemethod for algorithm comparisons based on comparing how models use the training data, leveraging the recently proposed datamodel representations [IPE+22].
Revised: We then present a two-stage method for algorithm comparisons based on comparing how models use the training data, leveraging the recently proposed datamodel representations [IPE+22].

Original: We demonstrate our framework through a case study comparing classiﬁers trainedon the W ATERBIRDS [SKH+20] dataset with/without ImageNet pre-training.
Revised: We demonstrate our framework through a case study comparing classifiers trained on the W ATERBIRDS [SKH+20] dataset with/without ImageNet pre-training.

Original:  Introduction
Revised:  Introduction

Original: How models perform out-of-distribution (OOD) is shaped in part by the biases of these models.
Revised: How models perform out-of-distribution (OOD) is shaped in part by the biases of these models.

Original: These biases are often inﬂuenced by the particular design choices we make.
Revised: These biases are often influenced by the particular design choices we make.

Original: For example, Hookeret al.
Revised: For example, Hooker et al.

Original: [HCC+19] ﬁnd that model compression—a common design choice prior to deployment—preserves test set accuracies but can signiﬁcantly degrade OOD performance.
Revised: [HCC+19] find that model compression—a common design choice prior to deployment— preserves test set accuracies but can significantly degrade OOD performance.

Original: In order to understandhow design choices—which together deﬁne a learning algorithm —impact model biases, we thusneed to be able to differentiate learning algorithms in a more ﬁne-grained way than accuracy alone.
Revised: In order to understand how design choices—which together define a learning algorithm —impact model biases, we thus need to be able to differentiate learning algorithms in a more fine-grained way than accuracy alone.

Original: Motivated by this observation, we develop a uniﬁed framework for comparing learning algorithms.
Revised: Motivated by this observation, we develop a unified framework for comparing learning algorithms.

Original: Our proposed framework comprises (a) a precise, quantitative deﬁnition of learning algorithm comparison; and (b) a concrete methodology for comparing any two algorithms.
Revised: Our proposed framework comprises (a) a precise, quantitative definition of learning algorithm comparison; and (b) a concrete methodology for comparing any two algorithms.

Original: For (a), we frame thealgorithm comparison problem as one of ﬁnding input transformations that distinguish the two algorithms.
Revised: For (a), we frame the algorithm comparison problem as one of finding input transformations that distinguish the two algorithms.

Original: This goal is different and more general than quantifying model similarity [DDS21; BNB21;
Revised: This goal is different and more general than quantifying model similarity [DDS21; BNB21;

Original: MRB18] or testing speciﬁc biases [HCK20].
Revised: MRB18] or testing specific biases [HCK20].

Original: For (b), we propose a two-stage method for comparingalgorithms in terms of how they use the training data .
Revised: For (b), we propose a two-stage method for comparing algorithms in terms of how they use the training data .

Original: In the ﬁrst stage of this method, we leverage datamodel representations [IPE+22] to ﬁnd weightedcombinations of training examples (which we call training directions ) that have disparate impact ontest-time behavior of models across learning algorithms.
Revised: In the first stage of this method, we leverage datamodel representations [IPE+22] to find weighted combinations of training examples (which we call training directions ) that have disparate impact on test-time behavior of models across learning algorithms.

Original: In the second stage, we ﬁlter the subpopulation of test examples that are most inﬂuenced by each identiﬁed training direction, then manuallyinspect them to infer a shared feature (e.g., all images contain a human in the background).
Revised: In the second stage, we filter the subpopulation of test examples that are most influenced by each identified training direction, then manually inspect them to infer a shared feature (e.g., all images contain a human in the background).

Original: We thentie this intuition back to our quantitative deﬁnition by designing a distinguishing feature transformation based on the shared feature (e.g., adding a human to the background).
Revised: We then tie this intuition back to our quantitative definition by designing a distinguishing feature transformation based on the shared feature (e.g., adding a human to the background).

Original: We demonstrate our framework through a case study comparing classiﬁers ﬁrst pre-trained on ImageNet [DDS+09; RDS+15] then ﬁne-tuned on W ATERBIRDS [SKH+20] with classiﬁers trained from scratch on W ATERBIRDS .
Revised: We demonstrate our framework through a case study comparing classifiers first pre-trained on ImageNet [DDS+09; RDS+15] then fine-tuned on W ATERBIRDS [SKH+20] with classifiers trained from scratch on W ATERBIRDS .

Original: Our results demonstrate that ImageNet pre-training reduces dependenceof models on some spurious correlations (e.g., yellow color → landbird) but also introduces newones (e.g., human face in the background → landbird).
Revised: Our results demonstrate that ImageNet pre-training reduces dependence of models on some spurious correlations (e.g., yellow color → landbird) but also introduces new ones (e.g., human face in the background → landbird).

Original:  Comparing learning algorithms
Revised:  Comparing learning algorithms

Original: In this section, we describe our (learning) algorithm comparison framework.
Revised: In this section, we describe our (learning) algorithm comparison framework.

Original: In Section 2.1, weformalize algorithm comparison as the task of identifying distinguishing transformations .
Revised: In Section 2.1, we formalize algorithm comparison as the task of identifying distinguishing transformations .

Original: These arefunctions that—when applied to test examples—signiﬁcantly and consistently change the predictions of one model class but not the other.
Revised: These are functions that—when applied to test examples—significantly and consistently change the predictions of one model class but not the other.

Original: In Section 2.2, we describe our method for identifyingdistinguishing feature transformations by comparing how each model class uses the training data.
Revised: In Section 2.2, we describe our method for identifying distinguishing feature transformations by comparing how each model class uses the training data.

Original: 2.1 Formalizing algorithm comparisons via distinguishing transformations
Revised: 2.1 Formalizing algorithm comparisons via distinguishing transformations

Original: The goal of algorithm comparison is to understand the ways in which two learning algorithms(trained on the same data distribution) differ in the models they yield.
Revised: The goal of algorithm comparison is to understand the ways in which two learning algorithms (trained on the same data distribution) differ in the models they yield.

Original: More speciﬁcally, we areinterested in comparing the model classes induced by the two learning algorithms:
Revised: More specifically, we are interested in comparing the model classes induced by the two learning algorithms:

Original: Deﬁnition 1 (Induced model class) .
Revised: Definition 1 (Induced model class) .

Original: Given an input space X , a label space Y , and a model space
Revised: Given an input space X , a label space Y , and a model space

Original: M ⊂ X → Y , a learning algorithm
Revised: M ⊂ X → Y , a learning algorithm

Original: A : ( X × Y ) ∗ → M is a function mapping a set of input-50 label pairs to a model.
Revised: A : ( X × Y ) ∗ → M is a function mapping a set of input- label pairs to a model.

Original: Fixing a data distribution
Revised: Fixing a data distribution

Original: D , the model class induced by algorithm A is the51 distribution over
Revised: D , the model class induced by algorithm A is the distribution over

Original: M that results from applying A to randomly sampled datasets from D .
Revised: M that results from applying A to randomly sampled datasets from D .

Original: We adopt the perspective that model classes differ insofar as they use different features to makepredictions.
Revised: We adopt the perspective that model classes differ insofar as they use different features to make predictions.

Original: To make this precise, we introduce the notion of distinguishing feature transformations :
Revised: To make this precise, we introduce the notion of distinguishing feature transformations :

Original: Deﬁnition 2 (Distinguishing feature transformation) .
Revised: Definition 2 (Distinguishing feature transformation) .

Original: Let A 1 , A 2 denote learning algorithms, S a55 dataset of input-label pairs, and
Revised: Let A 1 , A 2 denote learning algorithms, S a dataset of input-label pairs, and

Original: L a loss function.
Revised: L a loss function.

Original: Suppose M 1 and M 2 are models trained on dataset
Revised: Suppose M 1 and M 2 are models trained on dataset

Original: D using algorithms A 1 and A 2 respectively.
Revised: D using algorithms A 1 and A 2 respectively.

Original: Then, a ( (cid:15), δ ) -distinguishing feature transformation of57 M 1 with respect to M 2 is a function F :
Revised: Then, a ( ϵ, δ ) -distinguishing feature transformation of M 1 with respect to M 2 is a function F :

Original: X → X such that for some label y c ∈ Y ,
Revised: X → X such that for some label y c ∈ Y ,

Original: (cid:125)(cid:124)

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: Counterfactual effect of F on

Original: [Equation]
Revised: [Equation]

Original: where L i ( x, y )
Revised: where L i ( x, y )

Original: =
Revised: =

Original: L ( M i ( x ) , y ) , and the expectations above are taken over both inputs x and randomness in the learning algorithm.
Revised: L ( M i ( x ) , y ) , and the expectations above are taken over both inputs x and randomness in the learning algorithm.

Original: Intuitively, a distinguishing feature transformation is just a function F that, when applied to test datapoints, signiﬁcantly changes the predictions of one model class—but not the other—in a consistentway.
Revised: Intuitively, a distinguishing feature transformation is just a function F that, when applied to test data points, significantly changes the predictions of one model class—but not the other—in a consistent way.

Original: Deﬁnition 2 also immediately suggests a way to evaluate the effectiveness of a distinguishingfeature transformation.
Revised: Definition 2 also immediately suggests a way to evaluate the effectiveness of a distinguishing feature transformation.

Original: That is, given a hypothesis about how two algorithms differ (e.g., that modelstrained with A 1 are more sensitive to texture than those trained with A 2 ), one can design a corresponding transformation F (e.g., applying style transfer, as in [GRM+19]), and directly measure itsrelative effect on the two model classes.
Revised: That is, given a hypothesis about how two algorithms differ (e.g., that models trained with A 1 are more sensitive to texture than those trained with A 2 ), one can design a corresponding transformation F (e.g., applying style transfer, as in [GRM+19]), and directly measure its relative effect on the two model classes.

Original: Since our goal is to qualitatively understand the differences in the biases of the model classes, welook for distinguishing feature transformations that are informative , i.e., they (a) capture a featurethat naturally arises in the data distribution and (b) are semantically meaningful.
Revised: Since our goal is to qualitatively understand the differences in the biases of the model classes, we look for distinguishing feature transformations that are informative , i.e., they (a) capture a feature that naturally arises in the data distribution and (b) are semantically meaningful.

Original: 2.2 Identifying distinguishing feature transformations
Revised: 2.2 Identifying distinguishing feature transformations

Original: We now describe our two-stage method for comparing learning algorithms based on how they use thetraining data .
Revised: We now describe our two-stage method for comparing learning algorithms based on how they use the training data .

Original: We ﬁrst identify training directions , or weighted combinations of training examples,that impact test performance of models trained with one learning algorithm
Revised: We first identify training directions , or weighted combinations of training examples, that impact test performance of models trained with one learning algorithm

Original: but not the other.
Revised: but not the other.

Original: Then,we use a human-in-the-loop approach to extract semantically meaningful features and correspondingtransformations from these directions.
Revised: Then, we use a human-in-the-loop approach to extract semantically meaningful features and corresponding transformations from these directions.

Original: Stage I: An algorithm for ﬁnding distinguishing training directions.
Revised: Stage I: An algorithm for finding distinguishing training directions.

Original: First, we ﬁnd weightedcombinations of training examples that inﬂuence the predictions of models trained with one learning algorithm
Revised: First, we find weighted combinations of training examples that influence the predictions of models trained with one learning algorithm

Original: but not the other.
Revised: but not the other.

Original: Our algorithm leverages datamodel representations [IPE+22] andcomprises the following three steps:
Revised: Our algorithm leverages datamodel representations [IPE+22] and comprises the following three steps:

Original: Compute datamodels for each algorithm.
Revised: Compute datamodels for each algorithm.

Original: Given training set S and learning algorithm A , adatamodel (representation) for test example x i is a vector θ i ∈ R | S | , where θ ij measures the extentto which models trained with A depend  on the j -th training example to correctly classify examplex i .
Revised: Given training set S and learning algorithm A , a datamodel (representation) for test example x i is a vector θ i ∈ R | S | , where θ ij measures the extent to which models trained with A depend 2 on the j -th training example to correctly classify example x i .

Original: We compute two sets of datamodels— θ (1) and θ (2) —corresponding to model classes inducedby learning algorithms A 1 and A 2 respectively.2.
Revised: We compute two sets of datamodels— θ (1) and θ (2) —corresponding to model classes induced by learning algorithms A 1 and A 2 respectively.

Original: Compute residual datamodels.
Revised: Compute residual datamodels.

Original: Next, we compute a residual datamodel for each test example x i , which is the projection of the datamodel θ (1) i onto the null space of datamodel θ (2)
Revised: Next, we compute a residual datamodel for each test example x i , which is the projection of the datamodel θ (1) i onto the null space of datamodel θ (2)

Original: i , i
Revised: i , i.e.,

Original: e.,θ (1 \ 2) i = ˆ θ (1) i − (cid:104) ˆ θ (1) i , ˆ θ (2) i (cid:105)
Revised: [Equation]

Original: ˆ θ (2) i , where ˆ θ i
Revised: where ˆ θ i

Original: = θ i /(cid:107)
Revised: = θ i /

Original:  θ i (cid:107) 2 denotes the normalized version of datamodelθ i .
Revised: ∥ θ i ∥ 2 denotes the normalized version of datamodel θ i .

Original: Intuitively, the residual datamodels of algorithm
Revised: Intuitively, the residual datamodels of algorithm

Original: A 1 with respect to A 2 correspond to the training89 directions that inﬂuence A 1 after “projecting away” the component that also inﬂuences A 2 .3.
Revised: A 1 with respect to A 2 correspond to the training directions that influence A 1 after “projecting away” the component that also influences A 2 .

Original: Run principal component analysis.
Revised: Run principal component analysis.

Original: Finally, we use principal component analysis (PCA) to ﬁndthe highest-variance directions in the space of residual datamodels. 
Revised: Finally, we use principal component analysis (PCA) to find the highest-variance directions in the space of residual datamodels. That is, we run

Original:  Intuitively (deferring formalanalysis to Appendix B), we expect the returned principal components to be the most distinguishingtraining directions across the test set.
Revised: [Equation] to find the top ℓ principal components of the residual datamodels. Intuitively (deferring formal analysis to Appendix B), we expect the returned principal components to be the most distinguishing training directions across the test set.

Original: Stage II: Human-in-the-loop analysis.
Revised: Stage II: Human-in-the-loop analysis.

Original: With these distinguishing directions in hand, we use ahuman-in-the-loop analysis to identify informative distinguishing features transformations in threesteps.
Revised: With these distinguishing directions in hand, we use a human-in-the-loop analysis to identify informative distinguishing features transformations in three steps.

Original: First, given a principal component, we inspect the test examples whose residual datamodelsare most aligned with that component.
Revised: First, given a principal component, we inspect the test examples whose residual datamodels are most aligned with that component.

Original: We view these examples as representing subpopulations thatdepend most heavily on the training direction.
Revised: We view these examples as representing subpopulations that depend most heavily on the training direction.

Original: With this subpopulation, we then infer a distinguishing feature by visual inspectionand if needed
Revised: With this subpopulation, we then infer a distinguishing feature by visual inspectionand if needed, additional analysis.

Original: , additional analysis Finally, we design a distinguishingfeature transformation to counterfactually verify the effect of the inferred feature on model behavior.
Revised: Finally, we design a distinguishing feature transformation to counterfactually verify the effect of the inferred feature on model behavior.

Original: [Figure]
Revised: We illustrate our algorithm visually in the top half of Figure 1.

Original:  Applying the algorithm comparison framework
Revised:  Applying the algorithm comparison framework

Original: [Figure]

Original: [Figure]

Original: E x p l a i n e d V a r i a n c e u n d e r

Original: A l g o r i t

Original: h m A 2 (

Original: [Figure]

Original: ) ( W i t h o u t

Original: P r e - t r a i n i n g )

Original: [Figure]

Original: We now demonstrate our comparison framework through a case study comparing modelstrained with and without pre-training.
Revised: We now demonstrate our comparison framework through a case study comparing models trained with and without pre-training.

Original: More specifcially, we study the effect of ImageNetpre-training [DDS+09; KSL19] in the context of classiﬁers trained on the W ATERBIRDSdataset [SKH+20].
Revised: Setup. We study the effect of ImageNet pre-training [DDS+09; KSL19] in the context of classifiers trained on the W ATERBIRDS dataset 3 [SKH+20].

Original: Pre-training signiﬁcantly improves the “worst group” accuracy on  images wherethe background conﬂicts with the bird—but how does it impact the ﬁne-grained biases of the models?
Revised: ImageNet pre-training significantly improves the “worst group” accuracy on W ATERBIRDS images where the background conflicts with the bird—but how does it impact the fine-grained biases of the models?

Original: To study this, we compare  ResNet-50 classiﬁers ﬁrst pre-trained on ImageNet then ﬁne-tunedon W ATERBIRDS (algorithm A 1 ) with those trained from scratch on W ATERBIRDS (algorithm A 2 ).
Revised: To study this, we compare two classes of ResNet-50 models trained with the exact same settings modulo the use of ImageNet pre-training. That is, we consider the following two learning algorithms, with additional details in Appendix C.2. • Algorithm A 1 : Pre-training on ImageNet, followed by full-network finetuning on W ATERBIRDS data. Models trained this way attain 89 .

Revised: 1% average accuracy and 63 .

Original: Identifying distinguishing features.
Revised: 6% worst-group accuracy. • Algorithm A 2 : Training from scratch on W ATERBIRDS data. Corresponding models attain 63 . 9% average accuracy and 5 .

Revised: 7% worst-group accuracy on the test set.

Revised: Identifying distinguishing features.

Original: We compare algorithms A 

Revised: We compare algorithms A

Original: and A 2 using our method110 from Section 2.
Revised: and A 2 using our method from Section 2.

Original: The ﬁrst stage of this method ﬁnds the top l distinguishing training directions 
Revised: The first stage of this method finds the top l distinguishing training directions { v 1 , ..., v l } ∈ R | S | .

Original: AddPatch
Revised: As we will show (in Appendix B), for a given training direction v , the fraction of variance that v explains 4 in the datamodel representations { θ ( i ) x } captures the importance of the corresponding combination of training examples to model predictions for algorithm A i . Thus, we would hope for training directions that distinguish A 1 from A 2 to explain a high (resp., low) amount of the variance in datamodel representations of algorithm A 1 (resp., A 2 ). Figure 2 displays the identified distinguishing training directions.

Revised: On the left, we can see that the training directions distinguishing A 1 from A 2 (in green) indeed explain a significant amount of vari- ance in the datamodels of

Revised: A 1 but not in those of A 2 .

Revised: Visualizing the subpopulations corresponding to two of the distinguishing directions (Figure 2 right) suggests the following distinguishing features:

Revised: [Figure]

Revised: E x p l a i n e d V a r i a n c e u n d e r

Revised: A l g o r i t

Revised: h m A 2 ( % )

Revised: ( W i t h

Revised: o u t P r e - t r a i n i n g )

Revised: B

Revised: A

Revised: [Figure]

Revised: A

Revised: B

Revised: AddPatch

Original: Size20px
Revised: Size20px

Original: AddPatch
Revised: AddPatch

Original: Size30px
Revised: Size30px

Original: AddPatch
Revised: AddPatch

Original: Size40px  40 PatchSize [-−−
-]
Revised: Size40px  40 PatchSize [-−−
-]− 4 −

Original: P e r c e n t I n c r e a s e i n P r ( l a n d b i r d )
Revised: P e r c e n t I n c r e a s e i n P r ( l a n d b i r d )

Original: ImageNetPre-training With
Revised: ImageNetPre-training With

Original: Without
Revised: Without

Original: AddPatch
Revised: AddPatch

Original: Size80px
Revised: Size80px

Original: AddPatch
Revised: AddPatch

Original: Size100px
Revised: Size100px

Original: AddPatch
Revised: AddPatch

Original: Size120px100  120 PatchSize −
Revised: Size120px  120 PatchSize −

Original: P e r c e n t I n c r e a s e i n P r ( l a n d b i r d )
Revised: P e r c e n t I n c r e a s e i n P r ( l a n d b i r d )

Original: ImageNetPre-training With
Revised: ImageNetPre-training With

Original: Without
Revised: Without

Original: { v 1 , ..., v l } ∈ R | S | .

Original: As we will show (in Appendix B), for a training direction v , the fractionof variance that v explains in the datamodel representations { θ ( i ) x } captures the importance of thecorresponding combination of training examples to model predictions for algorithm A i .

Original: Thus, wewould hope for training directions that distinguish A 1 from A 2 to explain a high (resp., low) amountof the variance in datamodel representations of algorithm A 1 (resp.,

Original: A 2 ).

Original: Figure 1 displays the identiﬁed distinguishing training directions.

Original: On the left, we can see that thetraining directions distinguishing A 1 from A 2 (in green) indeed explain a signiﬁcant amount of vari- 118 ance in the datamodels of

Original: A 1 but not in those of A 2 .

Original: Visualizing the subpopulations corresponding 119 to two of the distinguishing directions (Figure 1 right) suggests the following distinguishing features:

Original: Yellow color : Direction A surfaces a subpopulation of test images that contain yellow “landbird”images.
Revised: Yellow color: Direction A surfaces a subpopulation of test images that contain yellow “landbird” images.

Original: This leads us to hypothesize that models trained from scratch spuriously relies on the coloryellow to predict the class “landbird,” whereas ImageNet-pretrained models do not.
Revised: This leads us to hypothesize that models trained from scratch spuriously relies on the color yellow to predict the class “landbird,” whereas ImageNet-pretrained models do not.

Original: Additional analysis supports this hypothesis, as training images that contain other yellow objects strongly inﬂuencethe predictions of models trained from scratch on this subpopulation (Appendix D).
Revised: Additional analysis supports this hypothesis, as training images that contain other yellow objects strongly influence the predictions of models trained from scratch on this subpopulation (Appendix D).

Original: To test thishypothesis, we design a feature transformation that adds a yellow square patch to images (Figure 2).
Revised: To test this hypothesis, we design a feature transformation that adds a yellow square patch to images (Figure 3).

Original: Human face: Direction B surfaces a subpopulation of “landbird” that have human faces in the background.
Revised: Human face: Direction B surfaces a subpopulation of “landbird” that have human faces in the background.

Original: This suggests that ImageNet pre-training introduces a spurious dependence on human facesto predict the label “landbird.”
Revised: This suggests that ImageNet pre-training introduces a spurious dependence on human faces to predict the label “landbird.”

Original: Additional analysis supports this hypothesis, as training imagescontaining human face(s) strongly inﬂuence the predictions of ImageNet-pretrained models on thissubpopulation (see Appendix D).
Revised: Additional analysis supports this hypothesis, as training images containing human face(s) strongly influence the predictions of ImageNet-pretrained models on this subpopulation (see Appendix D).

Original: To test this hypothesis, we design a feature transformation thatinserts patches of human faces to W ATERBIRDS image backgrounds ( Figure 2).
Revised: To test this hypothesis, we design a feature transformation that inserts patches of human faces to W ATERBIRDS image backgrounds (Figure 3).

Original: Findings.
Revised: Findings.

Original: In Figure 2, we compare the effect of the above feature transformations on models trainedwith and without ImageNet pre-training.
Revised: In Figure 3, we compare the effect of the above feature transformations on models trained with and without ImageNet pre-training.

Original: The results conﬁrm both of our hypotheses.
Revised: The results confirm both of our hypotheses.

Original: Adding ayellow square patch with varying size (20/30/40 px) to test images increased P ( “landbird” ) by4%/9%/12% for models trained from scratch but decreased P ( “landbird” ) for models pre-trainedon ImageNet.
Revised: Adding a yellow square patch with varying size (20/30/40 px) to test images increased P ( “landbird” ) by 4%/9%/12% for models trained from scratch but decreased P ( “landbird” ) for models pre-trained on ImageNet.

Original: Similarly, adding a human face patch to image backgrounds increased P ( “landbird” )by 2%/3%/4% for pre-trained models, but did not signiﬁcantly affect models trained from scratch.
Revised: Similarly, adding a human face patch to image backgrounds increased P ( “landbird” ) by 2%/3%/4% for pre-trained models, but did not significantly affect models trained from scratch.

Original: Furthermore, increasing the intensity (i.e., patch size) of these feature transformations further widensthe gap in sensitivity between the two model classes.
Revised: Furthermore, increasing the intensity (i.e., patch size) of these feature transformations further widens the gap in sensitivity between the two model classes.

Original: These differences verify that the feature transformations we constructed can distinguish the two learning algorithms as according to Deﬁnition 2.
Revised: These differences verify that the feature transformations we constructed can distinguish the two learning algorithms as according to Definition 2.

Revised: We defer additional analysis and details on feature transformations to Appendix E.

Revised: Connections to prior work.

Revised: This case study pinpoints how pre-training can alter the importance of different spurious correlations.

Original:   Conclusion
Revised: In particular, our results show that ImageNet pre-training reduces dependence on some spurious correlations (e.g., yellow color → landbird) but also introduces new ones (e.g., human face → landbird). Our findings thus shed light on two seemingly contradictory phenomena: pre-training can simultaneously improve robustness to spurious features [GML22; TLG+20] in target data as well as transfer new spurious correlations [SJI+22; NSZ20] from the pre-training dataset.  Conclusion

Original: We introduce a uniﬁed framework for ﬁne-grained comparisons of any two learning algorithms.
Revised: We introduce a unified framework for fine-grained comparisons of any two learning algorithms.

Original: Speciﬁcally, our framework compares models trained using two different algorithms in terms of howthe models rely on training data to make predictions.
Revised: Specifically, our framework compares models trained using two different algorithms in terms of how the models rely on training data to make predictions.

Original: In our case study, we applied our frameworkto pinpoint how pre-training can amplify or suppress speciﬁc spurious correlations.A Related work B B
Revised: In our case study, we applied our framework to pinpoint how pre-training can amplify or suppress specific spurious correlations.

Original: [-C Setup 
 C C. algorithms, and hyperparameters . .-]
Revised: [-C Setup 
 C C. algorithms, and hyperparameters . .-]Additional evaluation

Original: C C transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C. infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D
Revised: distinguishing feature

Original: E transformations[-
-]
Revised: transformations[-
-]

Original: F Miscellaneous results [-
-]
Revised: F Miscellaneous results [-
-]F

Revised: .1 Aggregate metric for algorithm comparison

Original: F
Revised: F

Revised: .2 Explained variance of residual datamodel principal components

Revised: F.3 Subpopulations surfaced by principal components of residual datamodels

Original: A Related work
Revised: A Related work

Original: We place our work into context with existing approaches to model and algorithm comparison inmachine learning.
Revised: We place our work into context with existing approaches to model and algorithm comparison in machine learning.

Original: In particular, we compare and contrast our approach to algorithm comparison with approaches to therelated problem of model comparison , where one tries to characterize the difference between two(usually ﬁxed) machine learning models.
Revised: In particular, we compare and contrast our approach to algorithm comparison with approaches to the related problem of model comparison , where one tries to characterize the difference between two (usually fixed) machine learning models.

Original: For simple models (e.g., sparse linear), we can directlycompare models in parameter space.
Revised: For simple models (e.g., sparse linear), we can directly compare models in parameter space.

Original: However, distances between more complex models such asdeep neural networks are much less meaningful.
Revised: However, distances between more complex models such as deep neural networks are much less meaningful.

Original: To this end, a long line of work has sought todesign methods for characterizing the differences between models:
Revised: To this end, a long line of work has sought to design methods for characterizing the differences between models:

Original: Representation-based comparison.
Revised: Representation-based comparison.

Original: A popular approach (particularly in the context of deeplearning) is to compare models using their internal representations .
Revised: A popular approach (particularly in the context of deep learning) is to compare models using their internal representations .

Original: Since the coordinates ofthese representations do not have a consistent interpretation, representation-based model comparison typically studies the degree to which different models’ representations can be aligned.
Revised: Since the coordinates of these representations do not have a consistent interpretation, representation-based model comparison typically studies the degree to which different models’ representations can be aligned.

Original: Methods based this approach include canonical correlation analysis (CCA) and variants [RGY+17;
Revised: Methods based this approach include canonical correlation analysis (CCA) and variants [RGY+17;

Original: MRB18; CKM+22], centered kernel alignment (CKA)
Revised: MRB18; CKM+22], centered kernel alignment (CKA)

Original: [KNL+19], graph-based methods [LYC+15;
Revised: [KNL+19], graph-based methods [LYC+15;

Original: CLH+21], and model stitching [CKM+21; BNB21].
Revised: CLH+21], and model stitching [CKM+21; BNB21].

Original: Prior works have used these methods to compare wide and deep neural networks [NRK21]; vision transformers and convolutional networks[RUK+21]; pre-trained and trained-from-scratch models [NSZ20]; and different language models[WBS+20].
Revised: Prior works have used these methods to compare wide and deep neural networks [NRK21]; vision transformers and convolutional networks [RUK+21]; pre-trained and trained-from-scratch models [NSZ20]; and different language models [WBS+20].

Original: Though they are often useful, prior work shows that representation-based similaritymeasures are not always statistically reliable for testing functional differences in models [DDS21].
Revised: Though they are often useful, prior work shows that representation-based similarity measures are not always statistically reliable for testing functional differences in models [DDS21].

Original: Our approach to algorithm comparison differs from these methods in both objective and implementation:
Revised: Our approach to algorithm comparison differs from these methods in both objective and implementation:

Original: • Learning algorithms rather than ﬁxed models : Rather than focusing on a single ﬁxed model, our objective in this 
Revised: • Learning algorithms rather than fixed models : Rather than focusing on a single fixed model, our objective in this paper is to compare the

Original: paper

Original: is to
Revised: class

Original: compare the class of models that result from a given
Revised: of models that result from a given

Original: learning algorithm.
Revised: learning algorithm

Original: In particular, we aim to ﬁnd only differences that arise from algorith- mic design choices, and not those that arise from the (sometimes signiﬁcant) variability intraining across random seeds [ZGK+21].
Revised: In particular, we aim to find only differences that arise from algorithmic design choices, and not those that arise from the (sometimes significant) variability in training across random seeds [ZGK+21].

Original: • Feature-based rather than similarity-based : Methods such as CCA and CKA focus on outputting a single score that reﬂects the overall similarity between two models.
Revised: • Feature-based rather than similarity-based : Methods such as CCA and CKA focus on outputting a single score that reflects the overall similarity between two models.

Original: On the otherhand, the goal of our framework is to ﬁnd ﬁne-grained differences in model behavior.
Revised: On the other hand, the goal of our framework is to find fine-grained differences in model behavior.

Original: Still,in Appendix F.1 we show that we can also use our method for more global comparisons,for instance by computing the average cosine similarity of the datamodel vectors.
Revised: Still, in Appendix F.1 we show that we can also use our method for more global comparisons, for instance by computing the average cosine similarity of the datamodel vectors.

Original: • Model-agnostic :
Revised: • Model-agnostic :

Original: Our framework is agnostic to type of model used and thus allows one toeasily compare models across learning algorithms—our method extends even to learningalgorithms that do not have explicit representations (e.g., decision trees and kernel methods).
Revised: Our framework is agnostic to type of model used and thus allows one to easily compare models across learning algorithms—our method extends even to learning algorithms that do not have explicit representations (e.g., decision trees and kernel methods).

Original: Example-level comparisons.
Revised: Example-level comparisons.

Original: An alternative method for comparing models is to compare theirpredictions directly.
Revised: An alternative method for comparing models is to compare their predictions directly.

Original: For example, Zhong et al.
Revised: For example, Zhong et al.

Original: [ZGK+21] compare predictions of small and largelanguage models (on a per-example level) to ﬁnd that larger models are not uniformly better acrossexample.
Revised: [ZGK+21] compare predictions of small and large language models (on a per-example level) to find that larger models are not uniformly better across example.

Original: Similarly, Mania et al.
Revised: Similarly, Mania et al.

Original: [MMS+19] study the agreement between models, i.e., how oftenthey output the same prediction on a per-example level.
Revised: [MMS+19] study the agreement between models, i.e., how often they output the same prediction on a per-example level.

Original: In another vein, Meding et al.
Revised: In another vein, Meding et al.

Original: [MBG+22]show that after removing impossible or trivial examples from test sets, different models exhibit morevariations in their predictions.
Revised: [MBG+22] show that after removing impossible or trivial examples from test sets, different models exhibit more variations in their predictions.

Original: Our framework also studies instance-level predictions, but ultimatelyconnects the results back to human-interpretable distinguishing features.
Revised: Our framework also studies instance-level predictions, but ultimately connects the results back to human-interpretable distinguishing features.

Original: Comparing feature attributions.
Revised: Comparing feature attributions.

Original: Finally, another line of work compares models in terms of howthey use features at test time.
Revised: Finally, another line of work compares models in terms of how they use features at test time.

Original: In the presence of a known set of features one can compute feature importances (e.g., using SHAP [LL17]) and compare them across models [WWZ+22].
Revised: In the presence of a known set of features one can compute feature importances (e.g., using SHAP [LL17]) and compare them across models [WWZ+22].

Original: In cases wherewe do not have access to high-level features, we can use instance-level explanation methods suchas saliency maps to highlight different parts of the input, but these methods generally do not helpat distinguishing models [DS22].
Revised: In cases where we do not have access to high-level features, we can use instance-level explanation methods such as saliency maps to highlight different parts of the input, but these methods generally do not help at distinguishing models [DS22].

Original: Furthermore, multiple evaluation metrics [AGM+18; 
Revised: Furthermore, multiple evaluation metrics [AGM+18; HEK+18;

Revised: SJN21] indicate that common instance-specific feature attribution methods can fail at accurately highlighting features learned by the model.

Original: Prior work on pre-training.
Revised: Prior work on pre-training.

Original: Our case study pinpoints how pre-training can alter the importance of different spurious correlations.
Revised: Our case study pinpoints how pre-training can alter the importance of different spurious correlations.

Original: In particular, our results show that ImageNet pre-training reduces dependence on some spurious correlations (e g., yellow color → landbird) but also introduces new ones (e g., human face → landbird).
Revised: In particular, our results show that ImageNet pre-training reduces dependence on some spurious correlations (e.g., yellow color → landbird) but also introduces new ones (e.g., human face → landbird).

Original: Our ﬁndings thus shed light on two seemingly contradic- tory phenomena: pre-training can simultaneously improve robustness to spurious features [GML22;
Revised: Our findings thus shed light on two seemingly contradictory phenomena: pre-training can simultaneously improve robustness to spurious features [GML22;

Original: TLG+20] in target data as well as transfer new spurious correlations [SJI+22; NSZ20] from the pre-training dataset
Revised: TLG+20] in target data as well as transfer new spurious correlations [SJI+22; NSZ20] from the pre-training dataset.

Original: B Algorithm analysis
Revised: B Algorithm analysis

Original: In the main paper, we applied our comparison framework to identify feature transformations thatdistinguished three pairs of learning algorithms.
Revised: In the main paper, we applied our comparison framework to identify feature transformations that distinguished three pairs of learning algorithms.

Original: Here, we describe in more detail the algorithmicstage of that framework, i.e., the stage whose purpose is to ﬁnd distinguishing training directions .
Revised: Here, we describe in more detail the algorithmic stage of that framework, i.e., the stage whose purpose is to find distinguishing training directions .

Original: To this end, we walk through each of the three steps of the algorithm presented in Section 2.2 andprovide intuition for how they identify distinguishing training directions.
Revised: To this end, we walk through each of the three steps of the algorithm presented in Section 2.2 and provide intuition for how they identify distinguishing training directions.

Original: B.1 A primer on datamodel representations
Revised: B.1 A primer on datamodel representations

Original: The ﬁrst step in our method is to compute datamodel vectors θ ( i ) j ∈ R | S | , one for each test input x j .
Revised: The first step in our method is to compute datamodel vectors θ ( i ) j ∈ R | S | , one for each test input x j .

Original: Each datamodel vector encodes the importance of individual training examples S to model’s lossat input x j when trained with learning algorithm
Revised: Each datamodel vector encodes the importance of individual training examples S to model’s loss at input x j when trained with learning algorithm

Original: A
Revised: A

Original: i .
Revised: i .

Original: More speciﬁcally, each vector corresponds tothe solution to a speciﬁc regression problem—these regression problems (explained below) form thebasis of our analysis.
Revised: More specifically, each vector corresponds to the solution to a specific regression problem—these regression problems (explained below) form the basis of our analysis.

Original: Setting up the regression problem.
Revised: Setting up the regression problem.

Original: Let us ﬁx a single learning algorithm A (being A 1 or A 2 ).
Revised: Let us fix a single learning algorithm A (being A 1 or A 2 ).

Original: For a given training set S = { x 1 , . . .
Revised: For a given training set S = { x 1 , . . .

Original: , x d } , a test input x , and subset S (cid:48) ⊂ S of the training set wedeﬁne the model output function as:f ( x, S (cid:48) )
Revised: , x d } , a test input x , and subset S ′ ⊂ S of the training set we define the model output function as: f [Equation]

Original:  = the loss after training a model on S (cid:48) and evaluating on x .
Revised: [Equation] ) = the loss after training a model on S ′ and evaluating on x .

Original: For example, f ( x, S (cid:48) ) can encode training a deep neural network on the subset S (cid:48) , then computingthe network’s correct-class margin on the input x .
Revised: For example, f ( x, S ′ ) can encode training a deep neural network on the subset S ′ , then computing the network’s correct-class margin on the input x .

Original: For a ﬁxed x , the corresponding regressionproblem is to predict the model output f ( x, S (cid:48) ) given a subset S (cid:48) .
Revised: For a fixed x , the corresponding regression problem is to predict the model output f ( x, S ′ ) given a subset S ′ .

Original: Datamodels.
Revised: Datamodels.

Original: Ilyas et al.
Revised: Ilyas et al.

Original: [IPE+22] show that—for deep neural networks trained on standard imageclassiﬁcation tasks—we can solve the regression problem above with a simple linear predictor.
Revised: [IPE+22] show that—for deep neural networks trained on standard image classification tasks—we can solve the regression problem above with a simple linear predictor.

Original: Morespeciﬁcally, they showed that
Revised: More specifically, they showed that

Original: [Equation]
Revised: [Equation]

Original: ) where θ x is a (learned) parameter vector (called the datamodel for x ), and 1 S (cid:48) ∈ { 0 , 1 } | S | is a binaryindicator vector of the set S (cid:48) , encoding whether each example of S is included in S (cid:48) , i.e.,
Revised: [Equation] where θ x is a (learned) parameter vector (called the datamodel for x ), and 1 S ′ ∈ { 0 , 1 } | S | is a binary indicator vector of the set S ′ , encoding whether each example of S is included in S ′ , i.e.,

Original: [Equation]
Revised: [Equation]

Original: S
Revised: S

Original: [Equation]
Revised: [Equation]

Original:  i [Equation]
Revised: = (cid:26)if x i ∈ S ′

Original: otherwise .
Revised: otherwise .

Original: In this way, datamodels consistitute linear approximations of model output functions.
Revised: In this way, datamodels consistitute linear approximations of model output functions.

Original: Datamodels as a representation space.
Revised: Datamodels as a representation space.

Original: While each datamodel is speciﬁc to an individual testinput, we can treat a collection of datamodels as embeddings or representations of test inputs { x j } jinto a common | S | -dimensional space.
Revised: While each datamodel is specific to an individual test input, we can treat a collection of datamodels as embeddings or representations of test inputs { x j } j into a common | S | -dimensional space.

Original: By comparing the representations, we can analyze the structure of the data—as used by the speciﬁc learning algorithm under study.
Revised: By comparing the representations, we can analyze the structure of the data—as used by the specific learning algorithm under study.

Original: Furthermore, these representations have a number of properties that make them useful for algorithm comparisons:
Revised: Furthermore, these representations have a number of properties that make them useful for algorithm comparisons:

Original: (a) Consistent basis: A datamodel representation for a ﬁxed train set S always has the same basis: coordinate i corresponds to the importance of the i -th training example.
Revised: (a) Consistent basis: A datamodel representation for a fixed train set S always has the same basis: coordinate i corresponds to the importance of the i -th training example.

Original: This consis-

Original: tency makes datamodels a convenient medium for algorithm comparisons, as representa- tions are automatically aligned across different learning algorithms, and even across mod- els that lack explicit representations (e g., decision trees).
Revised: This consistency makes datamodels a convenient medium for algorithm comparisons, as representations are automatically aligned across different learning algorithms, and even across models that lack explicit representations (e.g., decision trees).

Original: (b) Predictiveness: Datamodel vectors are causally predictive of model behavior.
Revised: (b) Predictiveness: Datamodel vectors are causally predictive of model behavior.

Original: That is, as Ilyas et al.
Revised: That is, as Ilyas et al.

Original: [IPE+22] show, we can use them to predict the counterfactual impact of removing or adding different training examples on model output for a given test example.As a result
Revised: [IPE+22] show, we can use them to predict the counterfactual impact of removing or adding different training examples on model output for a given test example.

Original: , any trends we ﬁnd across the datamodel representations come with a precise quantitative interpretation in terms of model outputs (to which we will come back to later in thise section)
Revised: As a result, any trends we find across the datamodel representations come with a precise quantitative interpretation in terms of model outputs (to which we will come back to later in thise section).

Original: (c) Density: Datamodel representations also have a high effective dimensionality : that is, one needs thousands of components to explain signiﬁcant fraction of variance, for instance, on C IFAR [--
-] [IPE+22].
Revised: (c) Density: Datamodel representations also have a high effective dimensionality : that is, one needs thousands of components to explain significant fraction of variance, for instance, on C IFAR [--
-]- [IPE+22].

Original: This suggests that datamodel representations encode ﬁne-grainedinformation about how each learning algorithm uses the training data, making them usefulfor uncovering subtle differences in model behavior.
Revised: This suggests that datamodel representations encode fine-grained information about how each learning algorithm uses the training data, making them useful for uncovering subtle differences in model behavior.

Original: B.2 Residual datamodels
Revised: B.2 Residual datamodels

Original: In Step 2 of our algorithm, using the two sets of datamodels { θ (1) } and { θ (2) } , we compute theresidual datamodel vectors:
Revised: In Step 2 of our algorithm, using the two sets of datamodels { θ (1) } and { θ (2) } , we compute the residual datamodel vectors:

Original: [Equation]
Revised: [Equation]

Original: (Note that this operation only makes sense because the two sets of datamodels live in the samevector space—see property (a) above.)
Revised: (Note that this operation only makes sense because the two sets of datamodels live in the same vector space—see property (a) above.)

Original: As we demonstrate, they correspond to datamodels for acertain residual model output function that we deﬁne below.
Revised: As we demonstrate, they correspond to datamodels for a certain residual model output function that we define below.

Original: Recall that our overarching goal is to ﬁnd training directions that that strongly inﬂuence modelstrained with learning algorithm A 

Revised: Recall that our overarching goal is to find training directions that that strongly influence models trained with learning algorithm A

Original: but not A 2 (or vice-versa) when classifying x .
Revised: but not A 2 (or vice-versa) when classifying x .

Original: More speciﬁcally,we want to ﬁnd training directions u that strongly inﬂuences f (1) ( x, S ) , while ignoring directionsthat also inﬂuence f (2) ( x, S ) .
Revised: More specifically, we want to find training directions u that strongly influences f (1) ( x, S ) , while ignoring directions that also influence f (2) ( x, S ) .

Original: In other words, what we care about is the “residual” of f (1) afterremoving the part that is correlated 3 with f (2) .
Revised: In other words, what we care about is the “residual” of f (1) after removing the part that is correlated 6 with f (2) .

Original: To capture this, consider the residual model outputfunction of A 1 relative to A 2 :
Revised: To capture this, consider the residual model output function of A 1 relative to A 2 :

Original: [Equation]
Revised: [Equation]

Original: where ρ f (1) f (2) is the correlation between two model output functions (across varying S ).
Revised: where ρ f (1) f (2) is the correlation between two model output functions (across varying S ).

Original: It turnsout that the datamodel for the residual output function is given precisely by the residual datamodeldeﬁned in Step 2 of our algorithm: the projection of one (normalized) datamodel representationinto the nullspace of the other representation.
Revised: It turns out that the datamodel for the residual output function is given precisely by the residual datamodel defined in Step 2 of our algorithm: the projection of one (normalized) datamodel representation into the nullspace of the other representation.

Original: That is, residual datamodels correspond to a linearapproximation of the residual model output:
Revised: That is, residual datamodels correspond to a linear approximation of the residual model output:

Original: [Equation]
Revised: [Equation]

Original: In summary, Step 2 of our procedure reduces understanding the differences in learning algorithms
Revised: In summary, Step 2 of our procedure reduces understanding the differences in learning algorithms

Original: A 1 and A 2 to analyzing their residual model outputs via residual datamodels.
Revised: A 1 and A 2 to analyzing their residual model outputs via residual datamodels.

Original: The residual data-415 models highlight directions that inﬂuence learning algorithm
Revised: The residual data- models highlight directions that influence learning algorithm

Original: A 1 after projecting away directions416 that also inﬂuence learning algorithm
Revised: A 1 after projecting away directions that also influence learning algorithm

Original: A 2 .
Revised: [Equation]

Original: B.3 Finding global trends with PCA
Revised: B.3 Finding global trends with PCA

Original: The output of Step 2 of our procedure is two set of residual datamodels—one set looking at theresidual of A 1 with respect to A 2 , and vice versa.
Revised: The output of Step 2 of our procedure is two set of residual datamodels—one set looking at the residual of A 1 with respect to A 2 , and vice versa.

Original: These residual datamodels capture directions that420 each algorithm is most sensitive to, though still on a per-example level.
Revised: These residual datamodels capture directions that each algorithm is most sensitive to, though still on a per-example level.

Original: On the other hand, our goalis to ﬁnd directions that each algorithm is most sensitive to on an aggregate level (as in Deﬁnition2).
Revised: On the other hand, our goal is to find directions that each algorithm is most sensitive to on an aggregate level (as in Definition 2).

Original: We now show how the ﬁnal step in our procedure, computing PCA on the residual datamodelrepresentations, achieves this goal.
Revised: We now show how the final step in our procedure, computing PCA on the residual datamodel representations, achieves this goal.

Original: For a given input x with datamodel θ , and for a training direction u ∈ R | S | , we can estimate theexample’s sensitivity to u as [( θ · u )] 2 —that is, the (estimated) effect on f ( x, S ) of upweighting anddownweighting the training samples according to u .
Revised: For a given input x with datamodel θ , and for a training direction u ∈ R | S | , we can estimate the example’s sensitivity to u as [( θ · u )] 2 —that is, the (estimated) effect on f ( x, S ) of upweighting and downweighting the training samples according to u .

Original: Since our goal is to ﬁnd a direction that theresidual model output function f (1 \ 2) is most sensitive in aggregate , our objective is
Revised: Since our goal is to find a direction that the residual model output function f (1 \ 2) is most sensitive in aggregate , our objective is

Original: [Equation]
Revised: [Equation]

Original: The direction that maximizes this objective is exactly the dominant principal component of the setof residual datamodels!
Revised: The direction that maximizes this objective is exactly the dominant principal component of the set of residual datamodels!

Original: Similarly, the top k principal components correspond to directions thatalgorithm A 1 is most sensitive to after accounting for A 2 .
Revised: Similarly, the top k principal components correspond to directions that algorithm A 1 is most sensitive to after accounting for A 2 .

Original: In summary, Step 3 of our procedure ﬁnds directions that strongly inﬂuence only one learning algorithm by examining directions of highest explained variance of the residual datamodel vectors.
Revised: In summary, Step 3 of our procedure finds directions that strongly influence only one learning algorithm by examining directions of highest explained variance of the residual datamodel vectors.

Original: C Experimental Setup
Revised: C Experimental Setup

Original: In this section, we outline the experimental setup—datasets, models, training algorithms, hyperparameters, and datamodels—used for our case study in Section 3.
Revised: In this section, we outline the experimental setup—datasets, models, training algorithms, hyperparameters, and datamodels—used for our case study in Section 3.

Original: C.1 Dataset
Revised: C.1 Dataset

Original: The Waterbirds dataset [SKH+20] consists of bird images taken from the CUB dataset [WBW+11]and pasted on backgrounds from the Places dataset [ZLK+17].
Revised: The Waterbirds dataset [SKH+20] consists of bird images taken from the CUB dataset [WBW+11] and pasted on backgrounds from the Places dataset [ZLK+17].

Original: The task here is to classify “waterbirds” and “landbirds” in the presence of spurious correlated “land” and “water” backgrounds inthe training data.
Revised: The task here is to classify “waterbirds” and “landbirds” in the presence of spurious correlated “land” and “water” backgrounds in the training data.

Original: Sagawa et al.
Revised: Sagawa et al.

Original: [SKH+20] introduce Waterbirds as a benchmark to evaluate modelsunder subpopulation shifts induced by spurious correlations.
Revised: [SKH+20] introduce Waterbirds as a benchmark to evaluate models under subpopulation shifts induced by spurious correlations.

Original: In our case study, we compare howmodels trained from scratch on Waterbirds data differ from ImageNet-pretrained models that areﬁne-tuned on Waterbirds data.
Revised: In our case study, we compare how models trained from scratch on Waterbirds data differ from ImageNet-pretrained models that are fine-tuned on Waterbirds data.

Original: Summary statistics of the W ATERBIRDS dataset are outlined in Table 1.
Revised: Summary statistics of the W ATERBIRDS dataset are outlined in Table 1.

Original: [Table]
Revised: [Table]

Original: C.2 Models, learning algorithms, and hyperparameters
Revised: C.2 Models, learning algorithms, and hyperparameters

Original: We use the standard ResNet50 architecture from the torchvision library.
Revised: We use the standard ResNet50 architecture from the torchvision library.

Original: We train models using
Revised: We train models using

Original: SGD with momentum 0 .
Revised: SGD with momentum 0 .

Original: 9 and weight decay 0 .
Revised: 9 and weight decay 0 .

Original: 0001 for a maximum of 50 epochs (and stop early ifthe training loss drops below 0 .01 ).
Revised: 0001 for a maximum of 50 epochs (and stop early if the training loss drops below 0 .

Original: For model selection, we choose the model checkpoint that hasthe maximum average accuracy on the validation dataset.
Revised: For model selection, we choose the model checkpoint that has the maximum average accuracy on the validation dataset.

Original: As in Sagawa et al.
Revised: As in Sagawa et al.

Original: [SKH+20], we do notuse data augmentation.
Revised: [SKH+20], we do not use data augmentation.

Original: In our case study on pre-training, we consider ImageNet pre-trained modelsfrom torchvision .
Revised: In our case study on pre-training, we consider ImageNet pre-trained models from torchvision .

Original: We consider models trained using the following algorithms:
Revised: We consider models trained using the following algorithms:

Original: • Algorithm A 1 (ImageNet pre-training) :
Revised: • Algorithm A 1 (ImageNet pre-training) :

Original: Models pre-trained on ImageNet are fully ﬁnetuned on Waterbirds data with a ﬁxed SGD learning rate 0 .
Revised: Models pre-trained on ImageNet are fully fine-tuned on Waterbirds data with a fixed SGD learning rate 0 .

Original: 005 and batch size 64 .
Revised: 005 and batch size 64 .

Original: Onaverage, models attain 89 .
Revised: On average, models attain 89 .

Original: 1% (non-adjusted) average test accuracy and 63 .
Revised: 1% (non-adjusted) average test accuracy and 63 .

Original: 9% worst-grouptest accuracy.• Algorithm A 2 (Training from scratch) : Models are trained from scratch (i.e., randominitialization) on Waterbirds data with SGD: initial learning rate 0 .
Revised: 9% worst-group test accuracy. • Algorithm A 2 (Training from scratch) : Models are trained from scratch (i e., random initialization) on Waterbirds data with SGD: initial learning rate 0 .

Original: 01 , batch size 64 , anda linear learning rate schedule ( 0 . {+
+}
Revised: 01 , batch size and a linear learning rate schedule ( 0 . {+
+}

Original: 2 × every 15 epochs).
Revised: × every 15 epochs).

Original: On average, models attain 63 .
Revised: On average, models attain 63 .

Original: 6%average test accuracy and 5 .
Revised: 6% average test accuracy and 5 .

Original: 7% worst-group test accuracy.
Revised: 7% worst-group test accuracy

Original: C.3 Datamodels
Revised: C.3 Datamodels

Original: Now, we provide additional details on datamodels which, we recall, are used in the ﬁrst stage of ouralgorithm comparison framework (see Section 2).
Revised: Now, we provide additional details on datamodels which, we recall, are used in the first stage of our algorithm comparison framework (see Section 2).

Original: Estimating linear datamodels.
Revised: Estimating linear datamodels.

Original: Recall from Appendix B that the datamodel vector for example x j , θ ( i ) j ∈ R | S | , encodes the importance of individual training examples S to model’s loss atexample x j when trained with algorithm A i .
Revised: Recall from Appendix B that the datamodel vector for example x j , θ ( i ) j ∈ R | S | , encodes the importance of individual training examples S to model’s loss at example x j when trained with algorithm A i .

Original: Concretely, given test example x j and training set466 S
Revised: Concretely, given test example x j and training set

Original:  = { x 1 , . . .
Revised: S = { x 1 , . . .

Original: , x d } , the datamodel θ j is a sparse linear model (or surrogate function) trained on thefollowing regression task:
Revised: , x d } , the datamodel θ j is a sparse linear model (or surrogate function) trained on the following regression task:

Original: For a training subset S (cid:48) ⊂ S , can we predict the correct-class marginf A ( x j ; S (cid:48) ) of a model trained on S (cid:48) with algorithm A ?
Revised: For a training subset S ′ ⊂ S , can we predict the correct-class margin f A ( x j ; S ′ ) of a model trained on S ′ with algorithm A ?

Original: This task can be naturally formulated asthe following supervised learning problem:
Revised: This task can be naturally formulated as the following supervised learning problem:

Original: Given a training set { ( S i , f A ( x ; S i )) } mi =1 of size m , thedatamodel θ j (for example x j ) is the solution to the following problem:
Revised: Given a training set { ( S i , f A ( x ; S i )) } mi =1 of size m , the datamodel θ j (for example x j ) is the solution to the following problem:

Original: [Equation]
Revised: [Equation]

Original: where 1 S i is a boolean vector that indicates whether examples in the training dataset x ∈ S belongto the training subset S i .
Revised: where 1 S i is a boolean vector that indicates whether examples in the training dataset x ∈ S belong to the training subset S i .

Original: Note that each datamodel training point ( S i , f A ( x j , S i )) is obtained by (a)training a model
Revised: Note that each datamodel training point ( S i , f A ( x j , S i )) is obtained by (a) training a model

Original: f (e.g., ResNet9) on a subset of data S i (e.g., randomly subsampled CIFAR data)and (b) evaluating the trained model’s output on example x j .
Revised: f (e.g., ResNet9) on a subset of data S i (e.g., randomly subsampled CIFAR data) and (b) evaluating the trained model’s output on example x j .

Original: Ilyas et al.
Revised: Ilyas et al.

Original: [IPE+22] demonstrate thatlinear datamodels can accurately predict outputs of deep image classiﬁers.
Revised: [IPE+22] demonstrate that linear datamodels can accurately predict outputs of deep image classifiers.

Original: Datamodel estimation hyperparameters.
Revised: Datamodel estimation hyperparameters.

Original: Recall that our algorithm comparison frameworkin Section 2 involves estimating two sets of datamodels { θ (1) } and { θ (2) } for learning algorithms
Revised: Recall that our algorithm comparison framework in Section 2 involves estimating two sets of datamodels { θ (1) } and { θ (2) } for learning algorithms

Original: A 1 and A 2 respectively.
Revised: A 1 and A 2 respectively.

Original: In our case study, we estimate two datamodels, θ (1) i and θ (2) i for everyexample x i in the test dataset.
Revised: In our case study, we estimate two datamodels, θ (1) i and θ (2) i for every example x i in the test dataset.

Original: Estimating these datamodels entail three design choices:
Revised: Estimating these datamodels entail three design choices:

Original: • Sampling scheme for train subsets: Like in Ilyas et al.
Revised: • Sampling scheme for train subsets: Like in Ilyas et al.

Original: [IPE+22], we use α -random subsets of the training data, where α denotes the subsampling fraction; we set α = 50% as itmaximizes sample efﬁciency (or model reuse) for empirical inﬂuence estimation [FZ20],which is equivalent to a variant of linear datamodels [IPE+22].• Sample size for datamodel estimation: Recall that a datamodel training set of size m corresponds to training m models (e.g., m ResNet18 models on C IFAR -10) on independentlysampled train subsets (or masks).
Revised: [IPE+22], we use α -random subsets of the training data, where α denotes the subsampling fraction; we set α = 50% as it maximizes sample efficiency (or model reuse) for empirical influence estimation [FZ20], which is equivalent to a variant of linear datamodels [IPE+22]. • Sample size for datamodel estimation: Recall that a datamodel training set of size m corresponds to training m models (e g., m ResNet18 models on C IFAR -10) on independently sampled train subsets (or masks).

Original: We estimate datamodels on W ATERBIRDS 50 k samples(or models) per learning algorithm; we make a validation split using 10% of these samples.
Revised: We estimate datamodels on W ATERBIRDS 50 k samples (or models) per learning algorithm; we make a validation split using 10% of these samples.

Original: • (cid:96) 1 sparsity regularization: We use cross-validation to select the sparsity regularizationparameter λ .
Revised: • ℓ 1 sparsity regularization: We use cross-validation to select the sparsity regularization parameter λ .

Original: Speciﬁcally , for each datamodel, we evaluate the MSE on a validation splitto search over k = 50 logarithmically spaced values for λ along the regularization path.
Revised: Specifically, for each datamodel, we evaluate the MSE on a validation split to search over k = 50 logarithmically spaced values for λ along the regularization path.

Original: As in [IPE+22], we then re-compute the datamodel on the entire dataset with the optimal λvalue and all m training examples.
Revised: As in [IPE+22], we then re-compute the datamodel on the entire dataset with the optimal λ value and all m training examples.

Original: C.4 Feature transformations
Revised: C.4 Feature transformations

Original: As discussed in Section 2, we counterfactually verify distinguishing features (inferred via humanin-the-loop analysis) by evaluating whether feature transformations change model behavior as hypothesized.
Revised: As discussed in Section 2, we counterfactually verify distinguishing features (inferred via humanin-the-loop analysis) by evaluating whether feature transformations change model behavior as hypothesized.

Original: Here, we describe the feature transformations used in Section 3 in more detail 4 .
Revised: Here, we describe the feature transformations used in Section 3 in more detail 7 .

Original: Designing feature transformations.
Revised: Designing feature transformations.

Original: We design feature transformations that modify examplesby adding a speciﬁc patch.
Revised: We design feature transformations that modify examples by adding a specific patch.

Original: We vary the intensity of patch-based transformations via patch sizek .Additional details speciﬁc to each feature transformation in our case study:
Revised: We vary the intensity of patch-based transformations via patch size k .Additional details specific to each feature transformation in our case study:

Original: • Yellow feature.
Revised: • Yellow feature.

Original: We add a k × k square yellow patch to the input.
Revised: We add a k × k square yellow patch to the input.

Original: • Human face feature.
Revised: • Human face feature.

Original: We add a k × k image of a human face to the input.
Revised: We add a k × k image of a human face to the input.

Original: To avoid occlusion with objects in the image foreground, we add the human face patch to the background.
Revised: To avoid occlusion with objects in the image foreground, we add the human face patch to the background.

Original: We make a bank of roughly 300 human faces using ImageNet face annotations [YYF+22]

Original: by (a)

Original: cropping

Original:  out human faces from ImageNet validation examples and (b) manually removing mislabeled, low-resolution, and unclear human face images.
Revised: We make a bank of roughly 300 human faces using ImageNet face annotations [YYF+22] by (a) cropping out human faces from ImageNet validation examples and (b) manually removing mislabeled, lowresolution, and unclear human face images.

Original: Evaluating feature transformations.
Revised: Evaluating feature transformations.

Original: As shown in Section 3, given two learning algorithms A 1and A 2 , we evaluate whether a feature transformation F changes predictions of models trained with
Revised: As shown in Section 3, given two learning algorithms Aand A 2 , we evaluate whether a feature transformation F changes predictions of models trained with

Original: A 1 and A 2 as hypothesized.
Revised: A 1 and A 2 as hypothesized.

Original: To evaluate the counterfactual effect of transformation F on model M ,509 we evaluate the extent to which applying F to input examples x increases the conﬁdence of modelsin a particular class y .
Revised: To evaluate the counterfactual effect of transformation F on model M , we evaluate the extent to which applying F to input examples x increases the confidence of models in a particular class y .

Original: In our experiments, we estimate this counterfactual effect by averaging overall test examples and over 500 models trained with each learning algorithm.
Revised: In our experiments, we estimate this counterfactual effect by averaging over all test examples and over 500 models trained with each learning algorithm.

Original: C.5 Training infrastructure
Revised: C.5 Training infrastructure

Original: Data loading.
Revised: Data loading.

Original: We use FFCV 5 [LIE+22], which removes the data loading bottleneck for smallermodels, gives a 3 - 4 × improvement in throughput (i.e., number of models a day per GPU).
Revised: We use FFCV 8 [LIE+22], which removes the data loading bottleneck for smaller models, gives a 3 - 4 × improvement in throughput (i.e., number of models a day per GPU).

Original: Datamodels regression.
Revised: Datamodels regression.

Original: In addition to FFCV , we use the fast-l1 package 6 —a SAGA-based
Revised: In addition to FFCV , we use the fast-l1 package 9 —a SAGA-based

Original: GPU solver for (cid:96) 1 -regularized regression—to parallelise datamodel estimation.
Revised: GPU solver for ℓ 1 -regularized regression—to parallelise datamodel estimation.

Original: Computing resources.
Revised: Computing resources.

Original: We train our models on a cluster of machines, each with 9 NVIDIA A100or V100 GPUs and 96 CPU cores.
Revised: We train our models on a cluster of machines, each with 9 NVIDIA A or V100 GPUs and 96 CPU cores.

Original: We also use half-precision to increase training speed.
Revised: We also use half-precision to increase training speed.

Original: D Additional human-in-the-loop analysis
Revised: D Additional human-in-the-loop analysis

Original: As outlined in Section 2, the second stage of our framework applies human-in-the-loop analysis toinfer distinguishing feature transformations from training directions extracted via PCA on residualdatamodels.
Revised: As outlined in Section 2, the second stage of our framework applies human-in-the-loop analysis to infer distinguishing feature transformations from training directions extracted via PCA on residual datamodels.

Original: In this section, we present additional human-in-the-loop analysis in order to substantiatethe distinguishing features inferred in our case study.
Revised: In this section, we present additional human-in-the-loop analysis in order to substantiate the distinguishing features inferred in our case study.

Original: D.1 Tools for Inferring distinguishing features from PCA subpopulations
Revised: D.1 Tools for Inferring distinguishing features from PCA subpopulations

Original: In this section, we outline additional human-in-the-loop tools that we use to analyze subpopulationssurfaced by principal components (PCs) of residual datamodels.
Revised: In this section, we outline additional human-in-the-loop tools that we use to analyze subpopulations surfaced by principal components (PCs) of residual datamodels.

Original: • Class-speciﬁc visual inspection.
Revised: • Class-specific visual inspection.

Original: As shown in Section 3, the subpopulation of test exam- ples whose datamodels have maximum projection onto PCs of residual datamodels largely belong to same class.
Revised: As shown in Section 3, the subpopulation of test examples whose datamodels have maximum projection onto PCs of residual datamodels largely belong to same class.

Original: So, a simple-yet-effective way to identify subpopulation-speciﬁc dis- tinguishing feature(s) is to just visually contrast the surfaced subpopulation from a set of[Table] randomly sampled examples that belong to the same class.
Revised: So, a simple-yet-effective way to identify subpopulation-specific distinguishing feature(s) is to just visually contrast the surfaced subpopulation from a set of randomly sampled examples that belong to the same class.

Original: [Table]

Original: • Relative inﬂuence of training examples.
Revised: • Relative influence of training examples.

Original: Given a subset of test examples S (cid:48) ⊂ S , can we

Original: [Table]

Original:  identify a set of training examples T (cid:48) ⊂ T that strongly inﬂuence predictions on S (cid:48) when 
Revised: Given a subset of test examples S ′ ⊂ S , can we identify a set of training examples T ′ ⊂ T that strongly influence predictions on S ′ when models are trained with algorithm

Original: [Table]

Original: models are trained with algorithm

Original: A 1 but not when trained with A 2 ?
Revised: A 1 but not when trained with A 2 ?

Original: Given datamodel

Original: [Table] representations { θ (1) i } for A 1 and { θ (1) i } for A 2 , we apply a two-step heuristic approach[Table] identify training examples with high inﬂuence on A 1 relative to A 2 : 
Revised: Given datamodel representations { θ (1) i } for A 1 and { θ (1) i } for A 2 , we apply a two-step heuristic approach identify training examples with high influence on A 1 relative to A 2 : – First, given learning algorithm A i and test subset S ′ , we estimate the aggregate (positive or negative) influence of training example x k on subset S by taking the absolute sum over the corresponding datamodel weights: (cid:80) j ∈ S ′

Original: [Table]

Original: – First, given learning algorithm A i and test subset S (cid:48) , we estimate the aggregate (pos-

Original: [Table] itive or negative) inﬂuence of training example x k on subset S by taking the absolute [Table] sum over the corresponding datamodel weights: (cid:80) j ∈ S (cid:48) | θ ( i ) jk | .
Revised: | θ ( i ) jk | .

Original: – Then, we take the absolute difference between the aggregate inﬂuence estimates oftraining example x k using θ (1) and θ (2) .
Revised: – Then, we take the absolute difference between the aggregate influence estimates of training example x k using θ (1) and θ (2) .

Original: This difference measures the relative inﬂuence of training example x k on predictions of test subset S when models are trainedwith algorithm
Revised: This difference measures the relative influence of training example x k on predictions of test subset S when models are trained with algorithm

Original: A 1 instead of algorithm
Revised: A 1 instead of algorithm

Original: A 2 .
Revised: A 2 .

Original: In our analysis, we (a) identify training examples that have top-most relative inﬂuenceestimates and then (b) visually contrast the subsets oftest examples (one for each learningalgorithm) that are most inﬂuenced by these training examples.
Revised: In our analysis, we ( a ) identify training examples that have top-most relative influence estimates and then (b) visually contrast the subsets of examples ) that are most influenced by these training examples.

Original: D.2 Case study: ImageNet pre-training
Revised: D.2 Case study: ImageNet pre-training

Original: Our case study on W ATERBIRDS data shows that ImageNet pre-training reduces dependence onthe “yellow color” feature, but introduces dependence the “human face” feature.
Revised: Our case study on W ATERBIRDS data shows that ImageNet pre-training reduces dependence on the “yellow color” feature, but introduces dependence the “human face” feature.

Original: We support theseﬁndings with relative inﬂuence analysis in Figure 3 and additional visual inspection in Figure
Revised: We support these findings with relative influence analysis in Figure 4 and additional visual inspection in Figure

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: E Additional evaluation of distinguishing feature transformations
Revised: E Additional evaluation of distinguishing feature transformations

Original: Distinguishing feature transformations, which we recall from Section 2, are functions that, whenapplied to data points, change the predictions of one model class—but not the other—in a consistentway.
Revised: Distinguishing feature transformations, which we recall from Section 2, are functions that, when applied to data points, change the predictions of one model class—but not the other—in a consistent way.

Original: In our case study, we design distinguishing feature transformations that counterfactually verifyfeatures that are identiﬁed via human-in-the-loop analysis.
Revised: In our case study, we design distinguishing feature transformations that counterfactually verify features that are identified via human-in-the-loop analysis.

Original: Our ﬁndings in Section 3 use featuretransformations to quantitatively measure the relative effect of the identiﬁed features on modelstrained with different learning algorithms.
Revised: Our findings in Section 3 use feature transformations to quantitatively measure the relative effect of the identified features on models trained with different learning algorithms.

Original: In this section, we present additional ﬁndings on featuretransformations for each case study:
Revised: In this section, we present additional findings on feature transformations for each case study:

Original: In Section 3, we showed that ﬁne-tuning ImageNet-pretrained ResNet50 models on W ATERBIRDSdata instead of training from scratch alters the relative importance of two spurious features: “yellowcolor” and “human face”.
Revised: In Section 3, we showed that fine-tuning ImageNet-pretrained ResNet50 models on W ATERBIRDS data instead of training from scratch alters the relative importance of two spurious features: “yellow color” and “human face”.

Original: In Figure 5, we show that both feature transformations alter the predictionsof ImageNet-pretrained ResNet18 and ImageNet-pretrained ResNet50 models in a similar way.
Revised: In Figure 6, we show that both feature transformations alter the predictions of ImageNet-pretrained ResNet18 and ImageNet-pretrained ResNet50 models in a similar way.

Original: [Figure]
Revised: [Figure]

Revised: r c e n t I n c r e a s e i n P r ( l a n db i r d )

Original: [Figure]
Revised: [Figure]

Revised: P e r c e n t I n c r e a s e i n P r ( l a n db i r d )

Original: [Figure]
Revised: [Figure]

Original: the effect of “yellow color” and “human face” feature transformations on models trained with and without ImageNet pre-training.

Original: In both cases, unlike ResNet50 models trained from scratch, ImageNet-pretrained ResNet18 and ResNet50 models are sensitive to the “human face” transformation but not to the “yellow color” transformation.
Revised: models trained from scratch, ImageNet-pretrained ResNet18 and ResNet50 models are sensitive to the “human face” transformation but not to the “yellow color” transformation.

Original: F Miscellaneous resultsF.1 Aggregate metric for algorithm comparison
Revised: F Miscellaneous results F.1 Aggregate metric for algorithm comparison

Original: As discussed in Appendix A, we can repurpose our framework as a similarity metric that quantiﬁes the similarity of models trained with different learning algorithms in a more global manner.
Revised: As discussed in Appendix A, we can repurpose our framework as a similarity metric that quantifies the similarity of models trained with different learning algorithms in a more global manner.

Original: Astraightforward approach to output a similarity score (or distribution) is to compute the cosine similarity of datamodel vectors.
Revised: A straightforward approach to output a similarity score (or distribution) is to compute the cosine similarity of datamodel vectors.

Original: More concretely, let θ (1) i and θ (2) i denote the datamodels of examplex i with respect to models trained using learning algorithms A 1{+
+} and A 2 .
Revised: More concretely, let θ (1) i and θ (2) i denote the datamodels of example x i with respect to models trained using learning algorithms A {+
+} and A 2 .

Original: Then, the cosine similarity570 between θ (1) i and θ (2) i measures the extent to which models trained with A 1 and A 2 depend on the571 same set of training examples to make predictions on example x i .
Revised: Then, the cosine similarity between θ (1) i and θ (2) i measures the extent to which models trained with A 1 and A 2 depend on the same set of training examples to make predictions on example x i .

Original: We apply this metric to our W ATERBIRDS case study.
Revised: We apply this metric to our W ATERBIRDS case study.

Original: Figure 6 plots the distribution of cosinesimilarity of datamodels for multiple learning algorithms over all test examples.
Revised: Figure 7 plots the distribution of cosine similarity of datamodels for multiple learning algorithms over all test examples.

Original: The plot showsthat ImageNet-pretrained ResNet50 models are, on average, more similar to ImageNet-pretrained
Revised: The plot shows that ImageNet-pretrained ResNet50 models are, on average, more similar to ImageNet-pretrained

Original: ResNet18 models than to ResNet50 models pretrained on synthetically generated data [BWW+21]and models trained from scratch. 
Revised: ResNet18 models than to ResNet50 models pretrained on synthetically generated data [BWW+21] and models trained from scratch. [Figure] [Figure] [Figure]

Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: Cosine similarity w.r.t ResNet9 w/ high SGD noise
Revised: similarity w.r.t ResNet9 w/ high SGD noise

Original: Learning algorithms (average cosine similarity)
Revised: Learning algorithms (average cosine similarity)

Original: Width-0.5x ResNet9 w/ high SGD noise (0.76)
Revised: Width-0.5x ResNet9 w/ high SGD noise (0.76)

Original: ResNet9 w/ low SGD noise (0.59)
Revised: ResNet9 w/ low SGD noise (0.59)

Original: F.2 Explained variance of residual datamodel principal components
Revised: F.2 Explained variance of residual datamodel principal components

Original: Recall from Appendix B that the fraction of variance in datamodel representations { θ ( i ) x } explainedby training direction v signiﬁes the importance of the direction (or, combination of training examples) to predictions of models trained with algorithm A i .
Revised: Recall from Appendix B that the fraction of variance in datamodel representations { θ ( i ) x } explained by training direction v signifies the importance of the direction (or, combination of training examples) to predictions of models trained with algorithm A i .

Original: In our case study in Section 3, we showthat the top 5 − 6 principal components (PCs) of residual datamodels θ (1 \ 2) correspond to trainingdirections that have high explained w.r.t.
Revised: In our case study in Section 3, we show that the top 5 − 6 principal components (PCs) of residual datamodels θ (1 \ 2) correspond to training directions that have high explained w.r.t.

Original: datamodels of algorithm A 1 but not A 2 , and vice versa.
Revised: datamodels of algorithm A 1 but not A 2 , and vice versa.

Original: Figure 7 shows that the top- 100 PCs of residual datamodel θ (1 \ 2) (resp., θ (2 \ 1) ) have more (resp. ,less) explained variance on datamodel θ (1) than on datamodel θ (2) .0.
Revised: Figure 8 shows that the top- 100 PCs of residual datamodel θ (1 \ 2) (resp., θ (2 \ 1) ) have more (resp., less) explained variance on datamodel θ (1) than on datamodel θ (2) .

Original: Explained Variance under Algorithm A 1 (%) ( With Augmentation) 0.
Revised: Explained Variance under Algorithm A 1 (%) ( With Augmentation) 0.

Original: E x p l a i n e d
Revised: E x p l a i n e d

Original: V a r i a n c e u n d e r
Revised: V a r i a n c e u n d e r

Original: A l g o r i t h m
Revised: A l g o r i t h m

Original: A 2 ( % ) ( W i t h o u t
Revised: A 2 ( % ) ( W i t h o u t

Original: A u g m e n t a t i o n )
Revised: A u g m e n t a t i o n )

Original: Living17 / Data augmentation
Revised: Living17 / Data augmentation

Original: PCs of q ( 1 \ 2 ) PCs of q ( 2 \ 1 )
Revised: PCs of q ( 1 \ 2 ) PCs of q ( 2 \ 1 )

Original: Line x=y
Revised: Line x=y

Original: Explained Variance under Algorithm A 1 (%) ( With ImageNet Pre-training)
Revised: Explained Variance under Algorithm A 1 (%) ( With ImageNet Pre-training)

Original: E x p l a i n e d
Revised: E x p l a i n e d

Original: V a r i a n c e u n d e r
Revised: V a r i a n c e u n d e r

Original: A l g o r i t h m
Revised: A l g o r i t h m

Original: A 2 ( % ) ( W i t h o u t
Revised: A 2 ( % ) ( W i t h o u t

Original: P r e - t r a i n i n g )
Revised: P r e - t r a i n i n g )

Original: Waterbirds / Pretraining
Revised: Waterbirds / Pretraining

Original: PCs of q ( 1 \ 2 ) PCs of q ( 2 \ 1 )
Revised: PCs of q ( 1 \ 2 ) PCs of q ( 2 \ 1 )

Original: Line x=y
Revised: Line x=y

Original: [Figure]
Revised: [Figure]

Original: Explained Variance under Algorithm A 1 (%) ( High SGD Noise)
Revised: Explained Variance under Algorithm A 1 (%) ( High SGD Noise)

Original: [Figure]
Revised: [Figure]

Revised: E x p l a i n e d

Original: V [Figure] [Figure]
Revised: V a r i a n c e u n d e r

Original: w
Revised: A l g o r i t h m A 2 ( %

Original: [Figure]
Revised: [Figure]

Revised: ( L o w S

Original: G D
Revised: G D

Original: N
Revised: N

Original: [Figure]
Revised: [Figure]

Revised: i

Revised: [Figure]

Original: e )
Revised: e )

Original: CIFAR10 / SGD hyperparameters
Revised: CIFAR10 / SGD hyperparameters

Original: PCs of q ( 1 \ 2 ) PCs of q ( 2 \ 1 )
Revised: PCs of q ( 1 \ 2 ) PCs of q ( 2 \ 1 )

Original: Line x=y
Revised: Line x=y

Original: F.3 Subpopulations surfaced by principal components of residual datamodels
Revised: F.3 Subpopulations surfaced by principal components of residual datamodels

Original: As outlined in Section 2, the human-in-the-loop stage of our framework involves extracting test datasubpopulations from principal components (PCs) of residual datamodels.
Revised: As outlined in Section 2, the human-in-the-loop stage of our framework involves extracting test data subpopulations from principal components (PCs) of residual datamodels.

Original: Speciﬁcally, these subpopulations correspond to test examples whose residual datamodel representations have the mostpositive (top- k ) and most negative (bottom- k ) projection onto a given PC.
Revised: Specifically, these subpopulations correspond to test examples whose residual datamodel representations have the most positive (top- k ) and most negative (bottom- k ) projection onto a given PC.

Original: Here, we show that thetop- k and bottom- k subpopulations corresponding to the top few PCs of residual datamodels considered in Section 3 surface test examples with qualitatively similar properties.
Revised: Here, we show that the top- k and bottom- k subpopulations corresponding to the top few PCs of residual datamodels considered in Section 3 surface test examples with qualitatively similar properties.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]