Original: M ODEL T RANSFERABILITY WITH R ESPONSIVE D ECISION S UBJECTS
Revised: M ODEL T RANSFERABILITY WITH R ESPONSIVE D ECISION S UBJECTS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: This paper studies model transferability when human decision subjects respond to a deployed machine learning model.
Revised: This paper studies model transferability when human decision subjects respond to a deployed machine learning model.

Original: In our setting, an agent or a user corresponds to a sample ( X, Y ) drawn from a distribution D and will face a model h and its classiﬁcation result h
Revised: In our setting, an agent or a user corresponds to a sample ( X, Y ) drawn from a distribution D and will face a model h and its classiﬁcation result h

Original: ( X ) .
Revised: ( X ) .

Original: Agents can modify X to adapt to h , which will incur a distribution shift on ( X, Y ) .
Revised: Agents can modify X to adapt to h , which will incur a distribution shift on ( X, Y ) .

Original: Therefore, when training h , the learner will need to consider the subsequently “induced” distribution when the output model is deployed.
Revised: Therefore, when training h , the learner will need to consider the subsequently “induced” distribution when the output model is deployed.

Original: Our formulation is motivated by applications where the deployed machine learning models interact with human agents, and will ultimately face responsive and interactive data distributions.
Revised: Our formulation is motivated by applications where the deployed machine learning models interact with human agents, and will ultimately face responsive and interactive data distributions.

Original: We formalize the discussions of the transferability of a model by studying how the model trained on the available source distribution (data) would translate to the performance on the induced domain.
Revised: We formalize the discussions of the transferability of a model by studying how the model trained on the available source distribution (data) would translate to the performance on the induced domain.

Original: We provide both upper bounds for the performance gap due to the induced domain shift, as well as lower bounds for the trade-offs that a classiﬁer has to suffer on either the source training distribution or the induced target distribution.
Revised: We provide both upper bounds for the performance gap due to the induced domain shift, as well as lower bounds for the trade-offs that a classiﬁer has to suffer on either the source training distribution or the induced target distribution.

Original: We provide further instantiated analysis for two popular domain adaptation settings with covariate shift and target shift .
Revised: We provide further instantiated analysis for two popular domain adaptation settings with covariate shift and target shift .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Decision makers are increasingly required to be transparent on their decision making to offer the “right to explanation” (Goodman & Flaxman, 2017; Selbst & Powles, 2018; Ustun et al., 2019) 1 .
Revised: Decision makers are increasingly required to be transparent on their decision making to offer the “right to explanation” (Goodman & Flaxman, 2017; Selbst & Powles, 2018; Ustun et al., 2019) 1 .

Original: Being transparent also invites potential adaptations from the population, leading to potential shifts.
Revised: Being transparent also invites potential adaptations from the population, leading to potential shifts.

Original: We are motivated by settings where the deployed machine learning models interact with human agents, which will ultimately face data distributions that reﬂect how human agents respond to the models.
Revised: We are motivated by settings where the deployed machine learning models interact with human agents, which will ultimately face data distributions that reﬂect how human agents respond to the models.

Original: For instance, when a model is used to decide loan applications, candidates may adapt their features based on the model speciﬁcation in order to maximize their chances of approval; thus the loan decision classiﬁer observes a data distribution caused by its own deployment (e.g., see Figurefor a demonstration).
Revised: For instance, when a model is used to decide loan applications, candidates may adapt their features based on the model speciﬁcation in order to maximize their chances of approval; thus the loan decision classiﬁer observes a data distribution caused by its own deployment (e.g., see Figurefor a demonstration).

Original: Similar observations can be articulated for application in insurance sector (i.e. developing policy s.t. customers’ behaviors might adapt to lower premium (Haghtalab et al., 2020)), education sector (i.e. developing courses when students are less incentivized to cheat (Kleinberg &
Revised: Similar observations can be articulated for application in insurance sector (i.e. developing policy s.t. customers’ behaviors might adapt to lower premium (Haghtalab et al., 2020)), education sector (i.e. developing courses when students are less incentivized to cheat (Kleinberg &

Original: Raghavan, 2020)) and so on.
Revised: Raghavan, 2020)) and so on.

Original: [Figure]
Revised: [Figure]

Original: This paper investigates model transferability when the underlying distribution shift is induced by the deployed model.
Revised: This paper investigates model transferability when the underlying distribution shift is induced by the deployed model.

Original: What we would like to have is some guarantee on the transferability of a classiﬁer — that is, how training on the available source distribution D S translates to performance on the induced domain D ( h ) , which depends on the model h being deployed.
Revised: What we would like to have is some guarantee on the transferability of a classiﬁer — that is, how training on the available source distribution D S translates to performance on the induced domain D ( h ) , which depends on the model h being deployed.

Original: A key concept in our setting is the induced risk , deﬁned as the error a model incurs on the distribution induced by itself:
Revised: A key concept in our setting is the induced risk , deﬁned as the error a model incurs on the distribution induced by itself:

Original: [Equation]
Revised: [Equation]

Original: Most relevant to the above formulation is the strategic classiﬁcation literature (Hardt et al., 2016a;
Revised: Most relevant to the above formulation is the strategic classiﬁcation literature (Hardt et al., 2016a;

Original: Chen et al., 2020b).
Revised: Chen et al., 2020b).

Original: In this literature, agents are modeled as rational utility maximizers and game theoretical solutions were proposed to characterize the induced risk.
Revised: In this literature, agents are modeled as rational utility maximizers and game theoretical solutions were proposed to characterize the induced risk.

Original: However, our results are motivated by the following challenges in more general scenarios:
Revised: However, our results are motivated by the following challenges in more general scenarios:

Original: • Modeling assumptions being restrictive In many practical situations, it is often hard to faithfully characterize agents’ utilities.
Revised: • Modeling assumptions being restrictive In many practical situations, it is often hard to faithfully characterize agents’ utilities.

Original: Furthermore, agents might not be fully rational when they response.
Revised: Furthermore, agents might not be fully rational when they response.

Original: All the uncertainties can lead to a far more complicated distribution change in ( X, Y ) , as compared to often-made assumptions that agents only change X
Revised: All the uncertainties can lead to a far more complicated distribution change in ( X, Y ) , as compared to often-made assumptions that agents only change X

Original: but not Y
Revised: but not Y

Original: (Chen et al., 2020b).
Revised: (Chen et al., 2020b).

Original: • Lack of access to response data
Revised: • Lack of access to response data

Original: Another relevant literature to our work is performative prediction (Perdomo et al., 2020).
Revised: Another relevant literature to our work is performative prediction (Perdomo et al., 2020).

Original: In performative prediction, one would often require knowing D ( h ) or having samples observed from D ( h ) through repeated experiments.
Revised: In performative prediction, one would often require knowing D ( h ) or having samples observed from D ( h ) through repeated experiments.

Original: We posit that machine learning practitioners may only have access to data from the source distribution during training
Revised: We posit that machine learning practitioners may only have access to data from the source distribution during training

Original: and although they anticipate changes in the population due to human agents’ responses, they cannot observe this new distribution until the model is actually deployed.
Revised: and although they anticipate changes in the population due to human agents’ responses, they cannot observe this new distribution until the model is actually deployed.

Original: Retraining being costly
Revised: Retraining being costly

Original: Even when samples from the induced data distribution are available, retraining the model from scratch may be impractical due to computational constraints.
Revised: Even when samples from the induced data distribution are available, retraining the model from scratch may be impractical due to computational constraints.

Original: The above observations motivate us to understand the transferability of a model trained on the source data to the domain induced by the deployment of itself.
Revised: The above observations motivate us to understand the transferability of a model trained on the source data to the domain induced by the deployment of itself.

Original: We study several fundamental questions:
Revised: We study several fundamental questions:

Original: • Source risk ⇒ Induced risk For a given model h , how different is Err D ( h ) ( h ) , the error on the distribution induced by h , from Err D S ( h ) :
Revised: • Source risk ⇒ Induced risk For a given model h , how different is Err D ( h ) ( h ) , the error on the distribution induced by h , from Err D S ( h ) :

Original: = P D S ( h ( X ) (cid:54) =
Revised: = P D S ( h ( X ) (cid:54) =

Original: Y ) , the error on the source distribution?
Revised: Y ) , the error on the source?

Original: • Induced risk ⇒ Minimum induced risk How much higher is Err D ( h ) ( h ) , the error on the induced distribution, than min h (cid:48)
Revised: • Induced risk ⇒ Minimum induced risk How much higher is Err D ( h ) ( h ) , the error on the induced distribution, than min h (cid:48)

Original: Err D ( h (cid:48) )
Revised: Err D ( h (cid:48) )

Original: ( h (cid:48) ) , the minimum achievable induced error?
Revised: ( h (cid:48) ) , the minimum achievable induced error?

Original: • Induced risk of source optimal ⇒ Minimum induced risk Of particular interest, and as a special case of the above,
Revised: • Induced risk of source optimal ⇒ Minimum induced risk Of particular interest, and as a special case of the above,

Original: how does Err D ( h ∗ S ) ( h ∗ S ) , the induced error of the optimal model trained on the source distribution h ∗ S :=  min h 
Revised: how does Err D ( h ∗ S ) ( h ∗ S ) , the induced error of the optimal model trained on the source distribution h ∗ S := arg min h Err D S ( h ) , compare to h ∗ T :=

Original: Err D S ( h ) , compare to min h Err D ( h ) ( h ) ?
Revised: arg min h Err D ( h ) ( h ) ?

Original: • Lower bound for learning tradeoffs What is the minimum error a model must incur on either the source distribution Err D S ( h ) or its induced distribution Err D ( h ) ( h ) ?
Revised: • Lower bound for learning tradeoffs What is the minimum error a model must incur on either the source distribution Err D S ( h ) or its induced distribution Err D ( h ) ( h ) ?

Original: For the ﬁrst three questions, we prove upper bounds on the additional error incurred when a model trained on a source distribution is transferred over to its induced domain.
Revised: For the ﬁrst three questions, we prove upper bounds on the additional error incurred when a model trained on a source distribution is transferred over to its induced domain.

Original: We also provide lower bounds for the trade-offs a classiﬁer has to suffer on either the source training distribution or the induced target distribution.
Revised: We also provide lower bounds for the trade-offs a classiﬁer has to suffer on either the source training distribution or the induced target distribution.

Original: We then show how to specialize our results to two popular domain adaptation settings: covariate shift (Shimodaira, 2000; Zadrozny, 2004; Sugiyama et al., 2007; 2008;
Revised: We then show how to specialize our results to two popular domain adaptation settings: covariate shift (Shimodaira, 2000; Zadrozny, 2004; Sugiyama et al., 2007; 2008;

Original: Zhang et al., 2013b) and target shift (Lipton et al., 2018; Guo et al., 2020;
Revised: Zhang et al., 2013b) and target shift (Lipton et al., 2018; Guo et al., 2020;

Original: Zhang et al., 2013b).
Revised: Zhang et al., 2013b).

Original: All omitted proofs can be found in the Appendix.
Revised: All omitted proofs can be found in the Appendix.

Original: 1. R ELATED WORKS
Revised: 1. R ELATED WORKS

Original: Most relevant to us are three topics: strategic classiﬁcation (Hardt et al., 2016a;
Revised: Most relevant to us are three topics: strategic classiﬁcation (Hardt et al., 2016a;

Original: Chen et al., 2020b;
Revised: Chen et al., 2020b;

Original: Dekel et al., 2010; Dong et al., 2018a; Chen et al., 2020a; Miller et al., 2020; Kleinberg & Raghavan, 2020), a recently proposed notion of performative prediction (Perdomo et al., 2020; Mendler-D¨unner et al., 2020), and domain adaptation (Jiang, 2008; Ben-David et al., 2010; Sugiyama et al., 2008;
Revised: Dekel et al., 2010; Dong et al., 2018; Chen et al., 2020a; Miller et al., 2020; Kleinberg & Raghavan, 2020), a recently proposed notion of performative prediction (Perdomo et al., 2020; Mendler-D¨unner et al., 2020), and domain adaptation (Jiang, 2008; Ben-David et al., 2010; Sugiyama et al., 2008;

Original: Zhang et al., 2019; Kang et al., 2019; Zhang et al., 2020).
Revised: Zhang et al., 2019; Kang et al., 2019; Zhang et al., 2020).

Original: Hardt et al.
Revised: Hardt et al.

Original: (2016a) pioneered the formalization of strategic behavior in classiﬁcation based on a sequential two-player game between agents and classiﬁers.
Revised: (2016a) pioneered the formalization of strategic behavior in classiﬁcation based on a sequential two-player game between agents and classiﬁers.

Original: Subsequently, Chen et al.
Revised: Subsequently, Chen et al.

Original: (2020b) addressed the question of repeatedly learning linear classiﬁers against agents who are strategically trying to game the deployed classiﬁers.
Revised: (2020b) addressed the question of repeatedly learning linear classiﬁers against agents who are strategically trying to game the deployed classiﬁers.

Original: Most of the existing literature focuses on ﬁnding the optimal classiﬁer by assuming fully rational agents (and by characterizing the equilibrium response).
Revised: Most of the existing literature focuses on ﬁnding the optimal classiﬁer by assuming fully rational agents (and by characterizing the equilibrium response).

Original: In contrast, we do not make these assumptions and primarily study the transferability when only having knowledge of source data.Perdomo [Table] [Table]
Revised: In contrast, we do not make these assumptions and primarily study the transferability when only having knowledge of source data.

Original: 2020) advocate minimizing the error E z ∼D

Original: [Table]

Original: θ ) [ (cid:96) ( θ ; z )] , which they call the performative prediction risk where θ is the model parameter to be optimized.

Original: This falls into the same category as induced risk, but the two concepts differ due to different requirements of knowing the distribution of a D ( θ ) .

Original: Our focus is on the transferability when learning on the source training data.

Original: In addition, we study speciﬁc domain adaptation setting which generally do not assume the knowledge of D ( θ ) (and particularly we will not assume the knowledge of the supervision/label information on the transferred domain).

Original: Relevantly, there are recent learning theoretical results on proving bounds for learning from strategically adapted data (Chen et al., 2020b;

Original: Dong et al., 2018b; Shavit et al., 2020; Sundaram et al., 2021).

Original: This line of works have developed solutions and results with performance guarantees w.r.t.

Original: to the optimal model under the strategic responses since they explicitly model the strategic behaviors.

Original: Our work is not on learning the optimal model by being aware of, predicting and modeling strategic responses.

Original: In comparison, our inquiry is on understanding how the trained model on the source data will transfer to its induced distribution.

Original: Our result was inspired by the transferability results in domain adaptations (Ben-David et al., 2010;
Revised: Our result was inspired by the transferability results in domain adaptations (Ben-David et al., 2010;

Original: Crammer et al., 2008; David et al., 2010).
Revised: Crammer et al., 2008; David et al., 2010).

Original: Later works examined speciﬁc domain adaptation models, such as covariate shift (Shimodaira, 2000; Zadrozny, 2004; Gretton et al., 2009;
Revised: Later works examined speciﬁc domain adaptation models, such as covariate shift (Shimodaira, 2000; Zadrozny, 2004; Gretton et al., 2009;

Original: Sugiyama et al., 2008; Zhang et al., 2013b;a) and target/label shift (Lipton et al., 2018;
Revised: Sugiyama et al., 2008; Zhang et al., 2013b;a) and target/label shift (Lipton et al., 2018;

Original: Azizzadenesheli et al., 2019).
Revised: Azizzadenesheli et al., 2019).

Original: A commonly established solution is to perform reweighted training on the source data, and robust and efﬁcient solutions have been developed to estimate the weights accurately (Sugiyama et al., 2008;
Revised: A commonly established solution is to perform reweighted training on the source data, and robust and efﬁcient solutions have been developed to estimate the weights accurately (Sugiyama et al., 2008;

Original: Zhang et al.
Revised: Zhang et al.

Original: , 2013b;a; Lipton et al., 2018;
Revised: , 2013b;a; Lipton et al., 2018;

Original: Guo et al., 2020).
Revised: Guo et al., 2020).

Original: Our work, at the ﬁrst sight, looks similar to several other area of studies.
Revised: Our work, at the ﬁrst sight, looks similar to several other area of studies.

Original: For instance, the notion of observing an “induced distribution” resembles similarity to the adversarial machine learning literature (Lowd & Meek, 2005; Huang et al., 2011; Vorobeychik & Kantarcioglu, 2018).
Revised: For instance, the notion of observing an “induced distribution” resembles similarity to the adversarial machine learning literature (Lowd & Meek, 2005; Huang et al., 2011; Vorobeychik & Kantarcioglu, 2018).

Original: One of the major differences between us and adversarial machine learning is the true label
Revised: One of the major differences between us and adversarial machine learning is the true label

Original: Y stays the same for the attacked feature while in our paper, both X and Y might change in the adapted distribution D ( h ) .
Revised: Y stays the same for the attacked feature while in our paper, both X and Y might change in the adapted distribution D ( h ) .

Original: In
Revised: In

Original: Appendix A.2, we provide detailed comparisons with some areas in domain adaptations, including domain generalization, adversarial attack and test-time adaptation.
Revised: Appendix A.2, we provide detailed comparisons with some areas in domain adaptations, including domain generalization, adversarial attack and test-time adaptation.

Revised: In particular, similar to domain generalization, one of the biggest challenge for our setting is the lack of access to data from the target distribution during training.

Original:  F ORMULATION
Revised:  F ORMULATION

Original: Suppose we are learning a parametric model h ∈ H for a binary classiﬁcation problem.
Revised: Suppose we are learning a parametric model h ∈ H for a binary classiﬁcation problem.

Original: Its training data set S :
Revised: Its training data set S :

Original: = { x i , y i } Ni =1 is drawn from a source distribution D S , where x i ∈ R d and y i ∈ {− 1 , +1 } .
Revised: = { x i , y i } Ni =1 is drawn from a source distribution D S , where x i ∈ R d and y i ∈ {− 1 , +1 } .

Original: However, h will then be deployed in a setting where the samples come from a test or target distribution
Revised: However, h will then be deployed in a setting where the samples come from a test or target distribution

Original: D T that can differ substantially from D S .
Revised: D T that can differ substantially from D S .

Original: Therefore instead of minimizing the prediction error on the source distribution Err DS ( h ) :
Revised: Therefore instead of minimizing the prediction error on the source distribution Err D

Original: = P D S ( h ( X ) (cid:54) =
Revised: S ( h ) := P D S ( h ( X ) (cid:54) =

Original: Y ) , the goal is to ﬁnd h ∗ that minimizes
Revised: Y ) , the goal is to ﬁnd h ∗ that minimizes

Original: Err D T ( h ) :=
Revised: Err D T ( h ) :=

Original: P D T ( h ( X ) (cid:54) = Y ) .
Revised: P D T ( h ( X ) (cid:54) = Y ) .

Original: This is often referred to as the domain adaptation problem , where typically, the transition from D S to D T is assumed to be independent of the model h being deployed.
Revised: This is often referred to as the domain adaptation problem , where typically, the transition from D S to D T is assumed to be independent of the model h being deployed.

Original: We consider a setting in which the distribution shift depends on h , or is thought of as being induced by h .
Revised: We consider a setting in which the distribution shift depends on h , or is thought of as being induced by h .

Original: We will use D ( h ) to denote the induced domain by h :
Revised: We will use D ( h ) to denote the induced domain by h :

Original: [Equation]
Revised: [Equation]

Original: Strictly speaking, the induced distribution is a function of both D S and h and should be better denoted by D S ( h ) .
Revised: Strictly speaking, the induced distribution is a function of both D S and h and should be better denoted by D S ( h ) .

Original: To ease the notation, we will stick with D ( h ) , but we shall keep in mind of its dependency of D S .
Revised: To ease the notation, we will stick with D ( h ) , but we shall keep in mind of its dependency of D S .

Original: For now, we do not restrict the dependency of D ( h ) of D and h , but later in Section 4 andwe will further instantiate D ( h ) under speciﬁc domain adaptation settings.
Revised: For now, we do not restrict the dependency of D ( h ) of D and h , but later in Section 4 andwe will further instantiate D ( h ) under speciﬁc domain adaptation settings.

Original: The challenge in the above setting is that when training h , the learner needs to carry the thoughts that
Revised: The challenge in the above setting is that when training h , the learner needs to carry the thoughts that

Original: D ( h ) should be the distribution it will be evaluated on and that the training cares about.
Revised: D ( h ) should be the distribution it will be evaluated on and that the training cares about.

Original: Formally, we deﬁne the induced risk of a classiﬁer h as the 0-1 error on the distribution h induces:
Revised: Formally, we deﬁne the induced risk of a classiﬁer h as the 0-1 error on the distribution h induces:

Original: [Equation]
Revised: [Equation]

Original: Denote by h ∗ T := arg min h ∈H Err D ( h ) ( h ) the classiﬁer with minimum induced risk.
Revised: Denote by h ∗ T := arg min h ∈H Err D ( h ) ( h ) the classiﬁer with minimum induced risk.

Original: More generally, when the loss may not be the 0-1 loss, we deﬁne the induced (cid:96) -risk as 
Revised: More generally, when the loss may not be the 0-1 loss, we deﬁne the induced (cid:96) -risk as [Equation] -risk

Original: [Equation]
Revised: [Equation]

Original: The induced risks will be the primary quantities that we are interested in minimizing.
Revised: The induced risks will be the primary quantities that we are interested in minimizing.

Original: The following additional notation will also be helpful:
Revised: The following additional notation will also be helpful

Original:  • Distributions of Y on a distribution D : D Y := P D ( Y = y ) 2 , and in particular D Y ( h ) :=
Revised: : • Distributions of Y on a distribution D : D Y := P D ( Y = y ) 2 , and in particular D Y ( h ) :

Original:  P D ( h ) ( Y = y ) , D Y | S := P D S ( Y = y ) .
Revised: = P D ( h ) ( Y = y ) , D Y | S := P D S ( Y = y ) .

Original: • Distribution of h on a distribution D : D h :=
Revised: • Distribution of h on a distribution D : D h :=

Original: P D ( h ( X ) = y ) , and in particular D h ( h ) :=
Revised: P D ( h ( X ) = y ) , and in particular D h ( h ) :=

Original: P D ( h ) ( h ( X ) = y ) , D h | S := P D S ( h ( X ) = y ) .
Revised: P D ( h ) ( h ( X ) = y ) , D h | S := P D S ( h ( X ) = y ) .

Original: • Marginal distribution of X for a distribution D : D X := P D ( X = x ) , and in particular D X ( h ) :
Revised: • Marginal distribution of X for a distribution D : D X := P D ( X = x ) , and in particular D X ( h ) :

Original: = P D ( h ) ( X = x ) , D X | S := P D S ( X = x ) 3 .
Revised: = P D ( h ) ( X = x ) , D X | S := P D S ( X = x ) 3 .

Original: • Total variation distancedeﬁned between D and D (cid:48) (Ali & Silvey, 1966): d TV ( D , D (cid:48) ) :
Revised: • Total variation distance (Ali & Silvey, 1966): d TV ( D , D (cid:48) ) :

Original: = sup O | P D ( O ) − P D (cid:48)
Revised: = sup O | P D ( O ) − P D (cid:48)

Original: ( O ) | .
Revised: ( O ) | .

Original:  2. E XAMPLES OF D ISTRIBUTION S HIFTS I NDUCED BY M ODEL D EPLOYMENT
Revised: The 2. E XAMPLES OF D ISTRIBUTION S HIFTS I NDUCED BY M ODEL D EPLOYMENT

Original: We provide two example models to demonstrate the use cases for the distribution shift models described in our paper.
Revised: We provide two example models to demonstrate the use cases for the distribution shift models described in our paper.

Original: We provide more details in Section 4.3 and Appendix B.2.
Revised: We provide more details in Section 4.3 and Section 5.3.

Original: Strategic Classiﬁcation
Revised: Strategic Classiﬁcation

Original: An example of distribution shift is when decision subjects perform strategic response to a decision rule.
Revised: An example of distribution shift is when decision subjects perform strategic response to a decision rule.

Original: It is well-known that when human agents are subject to a decision rule, they will adapt their features so as to get a favorable prediction outcome.
Revised: It is well-known that when human agents are subject to a decision rule, they will adapt their features so as to get a favorable prediction outcome.

Original: In the literature of strategic classiﬁcation, we say the human agents perform strategic adaptation (Hardt et al., 2016a).
Revised: In the literature of strategic classiﬁcation, we say the human agents perform strategic adaptation (Hardt et al., 2016a).

Original: It is natural to assume that the feature distribution before and after the human agents’ best response satisﬁes covariate shift : namely the feature distribution P ( X ) will change, but P ( Y | X ) , the mapping between Y and X , remain unchanged.
Revised: It is natural to assume that the feature distribution before and after the human agents’ best response satisﬁes covariate shift : namely the feature distribution P ( X ) will change, but P ( Y | X ) , the mapping between Y and X , remain unchanged.

Original: We use Figure 2 (Left) as a demonstrating of how distribution might shift for strategic response setting. In Section 4.3, we will use the  strategic classiﬁcation setup to verify our obtained results.
Revised: Notice that this is different from the assumption made in the classic strategic classiﬁcation setting Hardt et al. (2016a), which is to assume that the changes in the feature X does not change the underlying true qualiﬁcation Y . In our paper, we assume that changes in feature X could potential lead to changes in the true qualiﬁcation Y , and that the mapping between

Original:  X 1 X
Revised: Y and X remains the same before and after the adaptation.

Original: [Figure]
Revised: This is a commonly assumption made in a recent line of work on incentivizing improvement behaviors from human agents(see, e.g. Chen et al.

Original: X 2 X [Figure] X 3 X
Revised: (2020a)

Original: YY h ( X ) h ( X ) X ′ 1 X ′
Revised: ; Shavit et al.

Original: X ′ 2 X ′ X ′ 3 X ′ Y ′ Y ′ YY
Revised: We use Figure 2 (Left) as a demonstration of how distribution might shift for strategic response setting.

Original: X 1 X X 3 X X 2 Xh ( X ) h
Revised: In Section 4.3, we will use the strategic classiﬁcation setup to verify our obtained results.

Original: ( X )
Revised: [Equation]

Original: Y ′ Y ′ X ′ 1 X ′ X ′ 3 X ′ X ′ 2 X ′
Revised: [Figure]

Original: Replicator Dynamics Replicator dynamics is a commonly used model to study the evolution of an adopted “strategy” in evolutionary game theory (Tuyls et al., 2006; Friedman & Sinervo, 2016; Taylor & Jonker, 1978; Raab & Liu, 2021).
Revised: Replicator Dynamics Replicator dynamics is a commonly used model to study the evolution of an adopted “strategy” in evolutionary game theory (Tuyls et al., 2006; Friedman & Sinervo, 2016; Taylor & Jonker, 1978; Raab & Liu, 2021).

Original: The core notion of it is the growth or decline of the population of each strategy depends on its “ﬁtness”.
Revised: The core notion of it is the growth or decline of the population of each strategy depends on its “ﬁtness”.

Original: Consider the label Y
Revised: Consider the label Y

Original: = {− 1 , +1 } as the strategy, and the following behavioral response model to capture the induced target shift:
Revised: = {− 1 , +1 } as the strategy, and the following behavioral response model to capture the induced target shift:

Original: [Equation]
Revised: [Equation]

Original: In short, the change of the Y =
Revised: In short, the change of the Y =

Original: +1 population depends on how predicting Y = +1 “ﬁts” a certain utility function.
Revised: +1 population depends on how predicting Y = +1 “ﬁts” a certain utility function.

Original: For instance, the “ﬁtness” can take the form of the prediction accuracy of h for class +1 {+: Fitness ( Y = +1) : = P D S ( h ( X ) = +
 | Y = +1)+} .
Revised: For instance, the “ﬁtness” can take the form of the prediction accuracy of h for class +1 {+: Fitness ( Y = +1) : = P D S ( h ( X ) = +
 | Y = +1)+} .

Revised: Intuitively speaking, a higher “ﬁtness” describes more success of agents who adopted a certain strategy ( Y = − 1 or Y = +1 ).

Revised: Therefore, agents will imitate or replicate these successful peer agents by adopting the same strategy, resulting in an increase of the population ( P D ( h ) ( Y ) ).

Original: With assuming P ( X | Y ) stays unchanged, this instantiates one example of a speciﬁc induced target shift .
Revised: With assuming P ( X | Y ) stays unchanged, this instantiates one example of a speciﬁc induced target shift .

Original: We will specify the condition for target shift in Section 5.
Revised: We will specify the condition for target shift in Section 5.

Original: We use Figure 2 (Right) as a demonstrating of how distribution might shift for the replicator dynamic setting.
Revised: We use Figure 2 (Right) as a demonstrating of how distribution might shift for the replicator dynamic setting.

Original: In Appendix B.2, we will use a detailed replicator dynamics model to further instantiate our results.
Revised: In Section 5.3, we will use a detailed replicator dynamics model to further instantiate our results.

Original:  T RANSFERABILITY OF L EARNING TO I NDUCED D OMAINS
Revised:  T RANSFERABILITY OF L EARNING TO I NDUCED D OMAINS

Original: In this section, we ﬁrst provide upper bounds for the transfer error of a classiﬁer h (that is, the difference between Err D ( h ) ( h ) and Err D S ( h ) ), as well as between Err D ( h ) ( h ) and Err D ( h ∗ T ) ( h ∗ T ) .
Revised: In this section, we ﬁrst provide upper bounds for the transfer error of a classiﬁer h (that is, the difference between Err D ( h ) ( h ) and Err D S ( h ) ), as well as between Err D ( h ) ( h ) and Err D ( h ∗ T ) ( h ∗ T ) .

Original: We then provide lower bounds for max { Err D S ( h ) , Err D ( h ) ( h ) } , that is, the minimum error a model h must incur on either the source distribution D S or the induced distribution D ( h ) .
Revised: We then provide lower bounds for max { Err D S ( h ) , Err D ( h ) ( h ) } , that is, the minimum error a model h must incur on either the source distribution D S or the induced distribution D ( h ) .

Original: 3. U PPER B OUND
Revised: 3. U PPER B OUND

Original: We ﬁrst investigate upper bounds for the transfer errors.
Revised: We ﬁrst investigate upper bounds for the transfer errors.

Original: We begin by showing generic bounds, and further instantiate the bound for speciﬁc domain adaptation settings in Section 4 and 5 .
Revised: We begin by showing generic bounds, and further instantiate the bound for speciﬁc domain adaptation settings in Section 4 and 5 .

Original: We begin with answering a central question in domain adaptation:
Revised: We begin with answering a central question in domain adaptation:

Original: How does a model h trained on its training data set fare on the induced distribution D ( h ) ?
Revised: How does a model h trained on its training data set fare on the induced distribution D ( h ) ?

Original: To that end, deﬁne the minimum and h -dependent combined error of two distributions D and D (cid:48) as:
Revised: To that end, deﬁne the minimum and h -dependent combined error of two distributions D and D (cid:48) as:

Original: [Equation]
Revised: [Equation]

Original: and H -divergence asd H×H
Revised: and H -divergence as

Original: [Equation]
Revised: [Equation]

Original: D
Revised: |

Original: [Equation]
Revised: [Equation]

Original: D
Revised: The

Original: [Equation] = 2 sup h,h [Equation] The H -divergence is celebrated measure proposed in the domain adaptation literature (Ben-David et al., 2010) which will be useful for bounding the difference in errors of two classiﬁers.
Revised: H -divergence is celebrated measure proposed in the domain adaptation literature (Ben-David et al., 2010) which will be useful for bounding the difference in errors of two classiﬁers.

Original: Repeating classical arguments from Ben-David et al.
Revised: Repeating classical arguments from Ben-David et al.

Original: (2010), we can easily prove the following:
Revised: (2010), we can easily prove the following:

Original: Theorem 3.1 ( Source risk ⇒ Induced risk ) .
Revised: Theorem 3.1 ( Source risk ⇒ Induced risk ) .

Original: The difference between Err D ( h ) ( h ) and Err D S ( h ) is upper bounded by: Err D ( h ) ( h ) 
Revised: The difference between Err D ( h ) ( h ) and Err D S ( h ) is upper bounded by: Err D ( h ) ( h ) ≤ [Equation]

Original: ≤ Err D S ( h )
Revised: S [Equation]

Original: + λ D S→D
Revised: + [Equation] D S

Original: ( h ) + 12 d H×H
Revised: [Equation] d [Equation]

Original: ( D S , D ( h )) .
Revised: ( D S [Equation] D ( h )) .

Original: The transferability of a model h between Err D ( h ) ( h ) and Err D S ( h ) looks precisely the same as in the classical domain adaptation setting (Ben-David et al., 2010).
Revised: The transferability of a model h between Err D ( h ) ( h ) and Err D S ( h ) looks precisely the same as in the classical domain adaptation setting (Ben-David et al., 2010).

Original: Nonetheless, an arguably more interesting quantity in our setting to understand is the difference between the induced error of a given model h and the error induced by the optimal model: [Equation]
Revised: Nonetheless, an arguably more interesting quantity in our setting to understand is the difference between the induced error of a given model h and the error induced by the optimal model h ∗ T : Err D ( h ) ( h ) − Err D ( h ∗ T ) ( h ∗ T ) .

Original: We get the following bound, which differs from the one in Theorem 3.1:
Revised: We get the following bound, which differs from the one in Theorem 3.1:

Original: Theorem 3.2 ( Induced risk ⇒ Minimum induced risk ) .
Revised: Theorem 3.2 ( Induced risk ⇒ Minimum induced risk ) .

Original: The difference between Err D ( h ) ( h ) and 
Revised: The difference between Err D ( h ) ( h ) and Err D ( h

Original: Err D ( h ∗ T ) ( h ∗ T ) is upper bounded by: 
Revised: [Equation] T ) ( h ∗ T ) is upper bounded by: Err D

Original: [Equation]
Revised: [Equation]

Original: The above theorem informs us that the induced transfer error is bounded by the “average” achievable error on both distributions D ( h ) and D ( h ∗ T ) , as well as the H
Revised: The above theorem informs us that the induced transfer error is bounded by the “average” achievable error on both distributions D ( h ) and D ( h ∗ T ) , as well as the H

Original: × H divergence between the two distributions.
Revised: × H divergence between the two distributions.

Original: Reﬂecting on the difference between the bounds of Theorem 3.1 and Theorem 3.2, we see that the primary change is replacing the minimum achievable error λ with the average of λ and Λ .
Revised: Reﬂecting on the difference between the bounds of Theorem 3.1 and Theorem 3.2, we see that the primary change is replacing the minimum achievable error λ with the average of λ and Λ .

Original:  L OWER B OUND
Revised: 3. L OWER B OUND

Original: Now we provide a lower bound on the induced transfer error.
Revised: Now we provide a lower bound on the induced transfer error.

Original: We particularly want to show that at least one of the two errors Err D S ( h ) , and Err D ( h ) ( h ) , must be lower-bounded by a certain quantity.
Revised: We particularly want to show that at least one of the two errors Err D S ( h ) , and Err D ( h ) ( h ) , must be lower-bounded by a certain quantity.

Original: Theorem 3.3 ( Lower bound for learning tradeoffs ) .
Revised: Theorem 3.3 ( Lower bound for learning tradeoffs ) .

Original: Any model h must incur the following error on either the source or induced distribution: 
Revised: Any model h must incur the following error on either the source or induced distribution: max { Err D S ( h ) , Err D ( h ) ( h ) } ≥ d TV ( D

Original: [Equation]
Revised: [Equation]

Revised: S , D

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: d TV ( D h | S , D h ( h )).

Original: The proof leverages the triangle inequality of d TV .
Revised: The proof leverages the triangle inequality of d TV .

Original: This bound is dependent on h ; however, by the data processing inequality of d TV (and f -divergence functions in general) (Liese & Vajda, 2006), we have d TV ( D h | S , D h ( h ))
Revised: This bound is dependent on h ; however, by the data processing inequality of d TV (and f -divergence functions in general) (Liese & Vajda, 2006), we have d TV ( D h | S , D h ( h ))

Original: ≤ d TV ( D X | S , D X
Revised: ≤ d TV ( D X | S , D X

Original: ( h )) .
Revised: ( h )) .

Original: Applying this to Theorem 3.3 gives the following model-independent bound:
Revised: Applying this to Theorem 3.3 yields:

Original: Corollary 3.4.
Revised: Corollary 3.4.

Original: For any model h , max { Err D S ( h ) , [Equation]
Revised: For any model h , max { Err D S ( h ) , Err D ( h ) ( h ) } ≥ d TV ( D Y | S , D Y ( h )) − d TV

Original:  D [Equation] h [Equation]
Revised: ( D X | S , D X ( h )).

Original: H OW TO USE OUR BOUNDS
Revised: H OW TO USE OUR BOUNDS

Original: The upper and lower bounds we derived in the previous sections (Theorem 3.2 and Theorem 3.3) depend on the following two quantities either explicitly or implicitly: 1) the distribution D ( h ) induced by the deployment of the model h in question, and 2) the optimal target classiﬁer h ∗ T as well as the distribution D ( h ∗ T ) it induces.
Revised: The upper and lower bounds we derived in the previous sections (Theorem 3.2 and Theorem 3.3) depend on the following two quantities either explicitly or implicitly: 1) the distribution D ( h ) induced by the deployment of the model h in question, and 2) the optimal target classiﬁer h ∗ T as well as the distribution D ( h ∗ T ) it induces.

Original: The bounds may therefore seem to be of only theoretical interest, since in reality we generally cannot compute D ( h ) without actual deployment, let alone compute h ∗ T .
Revised: The bounds may therefore seem to be of only theoretical interest, since in reality we generally cannot compute D ( h ) without actual deployment, let alone compute h ∗ T .

Original: Thus in general it is unclear how to compute the value of these bounds.
Revised: Thus in general it is unclear how to compute the value of these bounds.

Original: Nevertheless, our bounds can still be useful and informative in the following ways:
Revised: Nevertheless, our bounds can still be useful and informative in the following ways:

Original: General modeling framework with ﬂexible hypothetical shifting models The bounds can be evaluated if the decision maker has a particular shift model in mind, which speciﬁes how the population would adapt to a model.
Revised: General modeling framework with ﬂexible hypothetical shifting models The bounds can be evaluated if the decision maker has a particular shift model in mind, which speciﬁes how the population would adapt to a model.

Original: A common special case is when the decision maker posits an individual-level agent response model (e.g. the strategic agent (Hardt et al., 2016a) - we demonstrate how to evaluate in Section 4.3).
Revised: A common special case is when the decision maker posits an individual-level agent response model (e.g. the strategic agent (Hardt et al., 2016a) - we demonstrate how to evaluate in Section 4.3).

Original: In these cases, the H -divergence can be consistently estimated from ﬁnite samples of the population (Wang et al., 2005), allowing the decision maker to estimate the performance gap of a given h without deploying it.
Revised: In these cases, the H -divergence can be consistently estimated from ﬁnite samples of the population (Wang et al., 2005), allowing the decision maker to estimate the performance gap of a given h without deploying it.

Original: The general bounds provided can thus be viewed as a framework by which specialized, computationally tractable bounds can be derived.
Revised: The general bounds provided can thus be viewed as a framework by which specialized, computationally tractable bounds can be derived.

Original: Estimate the optimal target classiﬁer h ∗ T from a set of imperfect models Secondly, when the decision maker has access to a set of imperfect models ˜ h 1 , ˜ h 2 · · · ˜ h t ∈ H T that will predict a range of possible shifted distribution D (˜ h 1 ) , · · · D (˜ h t ) ∈ D T and a range of possibly optimal target distribution h T ∈ H T , the bounds on h ∗ T can be further instantiated by calculating the worst case in this predicted set 4 :
Revised: Estimate the optimal target classiﬁer h ∗ T from a set of imperfect models Secondly, when the decision maker has access to a set of imperfect models ˜ h 1 , ˜ h 2 · · · ˜ h t ∈ H T that will predict a range of possible shifted distribution D (˜ h 1 ) , · · · D (˜ h t ) ∈ D T and a range of possibly optimal target distribution h T ∈ H T , the bounds on h ∗ T can be further instantiated by calculating the worst case in this predicted set 4 :

Original: [Equation]
Revised: [Equation]

Original: In addition, the challenge we are facing in this paper also shed lights on the danger of directly applying existing standard domain adaptation techniques when the shifting is caused by the deployment of the classiﬁer itself, since the bound will depend on the resulting distribution as well. 
Revised: In addition, the challenge we are facing in this paper also shed lights on the danger of directly applying existing standard domain adaptation techniques when the shifting is caused by the deployment of the classiﬁer itself, since the bound will depend on the resulting distribution as well. We add discussions on the tightness of our theoretical bounds in Appendix G.

Original:  C OVARIATE S HIFT
Revised:  C OVARIATE S HIFT

Original: In this section, we focus on a particular domain adaptation setting known as covariate shift , in which the distribution of features changes, but the distribution of labels conditioned on features does not:
Revised: In this section, we focus on a particular domain adaptation setting known as covariate shift , in which the distribution of features changes, but the distribution of labels conditioned on features does not:

Original: [Equation]
Revised: [Equation]

Original: Thus with covariate shift, we have
Revised: Thus with covariate shift, we have

Original: [Equation]
Revised: [Equation]

Original: Let ω x ( h ) :
Revised: Let ω x ( h ) :

Original: = P D ( h ) ( X = x ) P D S ( X = x ) be the importance weight at x , which characterizes the amount of adaptation induced by h at instance x .
Revised: = P D ( h ) ( X = x ) P D S ( X = x ) be the importance weight at x , which characterizes the amount of adaptation induced by h at instance x .

Original: Then for any loss function (cid:96) we have
Revised: Then for any loss function (cid:96) we have

Original: Proposition 4.1 (Expected Loss on D ( h ) ).
Revised: Proposition 4.1 (Expected Loss on D ( h ) )

Original: E D ( h ) [ (cid:96) ( h ; X, Y )]

Original: =

Original: E

Original: [Equation]
Revised: [Equation]

Original: S [ ω

Original: [Equation]

Original: [Equation]

Original: · (cid:96) ( h

Original: [Equation] x, y )] . The above derivation was not new and offered the basis for performing importance reweighting when learning under coviarate shift (Sugiyama et al., 2008).
Revised: The above derivation was not new and offered the basis for performing importance reweighting when learning under coviarate shift (Sugiyama et al., 2008).

Original: The particular form informs us that ω x ( h ) controls the generation of D ( h ) and encodes its dependency of both D S and h , and is critical for deriving our results below.
Revised: The particular form informs us that ω x ( h ) controls the generation of D ( h ) and encodes its dependency of both D S and h , and is critical for deriving our results below.

Original: 4. U PPER B OUND
Revised: 4. U PPER B OUND

Original: We now derive an upper bound for transferability under covariate shift.
Revised: We now derive an upper bound for transferability under covariate shift.

Original: We will focus particularly on the optimal model trained on the source data D S , which we denote as h ∗ S :=
Revised: We will focus particularly on the optimal model trained on the source data D S , which we denote as h ∗ S :=

Original: arg min h ∈H Err S ( h ) .
Revised: arg min h ∈H Err S ( h ) .

Original: Recall that the classiﬁer with minimum induced risk is denoted as h ∗ T :=
Revised: Recall that the classiﬁer with minimum induced risk is denoted as h ∗ T :=

Original: arg min h ∈H Err D ( h ) ( h ) .
Revised: arg min h ∈H Err D ( h ) ( h ) .

Original: We can upper bound the difference between h ∗ S and h ∗ T as follows:
Revised: We can upper bound the difference between h ∗ S and h ∗ T as follows:

Original: Theorem 4.2 (Suboptimality of h ∗ S ) .
Revised: Theorem 4.2 (Suboptimality of h ∗ S ) .

Original: Let X be distributed according to D S .
Revised: Let X be distributed according to D S .

Original: We have:
Revised: We have:

Original: [Equation]
Revised: [Equation]

Original: This result can be interpreted as follows: h ∗ T incurs an irreducible amount of error on the source data set, represented by (cid:112)
Revised: This result can be interpreted as follows: h ∗ T incurs an irreducible amount of error on the source data set, represented by (cid:112)

Original: Err D S ( h ∗ T ) .
Revised: Err D S ( h ∗ T ) .

Original: Moreover, the difference in error between h ∗ S and h ∗ T is at its maximum when the two classiﬁers induce adaptations in “opposite” directions; this is represented by the sum of the standard deviations of their importance weights,
Revised: Moreover, the difference in error between h ∗ S and h ∗ T is at its maximum when the two classiﬁers induce adaptations in “opposite” directions; this is represented by the sum of the standard deviations of their importance weights,

Original: (cid:112) Var ( ω X ( h ∗ S ))
Revised: (cid:112) Var ( ω X ( h ∗ S ))

Original: + (cid:112) Var ( ω X ( h ∗ T )) .
Revised: + (cid:112) Var ( ω X ( h ∗ T )) .

Original: 4. L OWER B OUND
Revised: 4. L OWER B OUND

Original: Recall from Theorem 3.3, for the general setting, it is unclear whether the lower bound is strictly positive or not.
Revised: Recall from Theorem 3.3, for the general setting, it is unclear whether the lower bound is strictly positive or not.

Original: In this section, we provide further understanding for when the lower bound d
Revised: In this section, we provide further understanding for when the lower bound d

Original: [Equation]
Revised: [Equation]

Original: is indeed positive under covariate shift.
Revised: is indeed positive under covariate shift.

Original: Under several assumptions, our previously provided lower bound in Theorem 3.3 is strictly positive with covariate shift.
Revised: Under several assumptions, our previously provided lower bound in Theorem 3.3 is strictly positive with covariate shift.

Original: Assumption4.3. | E X
Revised: Assumption

Original: [Equation]
Revised: [Equation]

Original:  + ( h )[Equation] = [Equation]
Revised: where X + ( h ) = { x : ω x ( h ) ≥ 1 } and X − ( h ) =

Original: [Equation] X ( h )] | .
Revised: { x : ω x ( h ) < 1 } .

Original: This assumption states that increased ω x ( h ) value points are more likely to have positive labels.
Revised: This assumption states that increased ω x ( h ) value points are more likely to have positive labels.

Original: Assumption4.4.
Revised: Assumption

Original: [Equation]
Revised: [Equation]

Original: This assumption states that increased ω x ( h ) value points are more likely to be classiﬁed as positive.
Revised: This assumption states that increased ω x ( h ) value points are more likely to be classiﬁed as positive.

Original: Assumption 4.5.
Revised: Assumption [Equation]

Original: [Equation]
Revised: [Equation]

Original: This assumption is stating that for a classiﬁer h , within all h ( X ) =
Revised: This assumption is stating that for a classiﬁer h , within all h ( X ) =

Original: +1 or h ( X ) = − 1 , a higher P
Revised: +1 or h ( X ) = − 1 , a higher P

Original: D ( Y = +

Revised: D ( Y = +

Original: | X = x ) associates with a higher ω x ( h ) .
Revised: | X = x ) associates with a higher ω x ( h ) .

Original: Theorem 4.6.
Revised: Theorem 4.6.

Original: Assuming 4.3 - 4.5, the following lower bound is strictly positive for covariate shift: 
Revised: Assuming 4.3 - 4.5, the following lower bound is strictly positive for covariate shift: [Equation] >

Original: [Equation]
Revised: [Equation]

Original: 4. E XAMPLE U SING S TRATEGIC C LASSIFICATION
Revised: 4. E XAMPLE U SING S TRATEGIC C LASSIFICATION

Original: As introduced in Section 2.1, we consider a setting caused by strategic response in which agents are classiﬁed by and adapt to a binary threshold classiﬁer.
Revised: As introduced in Section 2.1, we consider a setting caused by strategic response in which agents are classiﬁed by and adapt to a binary threshold classiﬁer.

Original: In particular, each agent is associated with a d dimensional continuous feature x ∈ R d and a binary true qualiﬁcation y ( x ) ∈ {− 1 , +1 } , where y ( x ) is a function of the feature vector x .
Revised: In particular, each agent is associated with a d dimensional continuous feature x ∈ R d and a binary true qualiﬁcation y ( x ) ∈ {− 1 , +1 } , where y ( x ) is a function of the feature vector x .

Original: Consistent with the literature in strategic classiﬁcation (Hardt et al., 2016a), a simple case where after seeing the threshold binary decision rule h ( x ) = 2 · 1 [ x ≥ τ h ] − 1 , the agents will best response to it by maximizing the following utility function: u ( x, x (cid:48) )
Revised: Consistent with the literature in strategic classiﬁcation (Hardt et al., 2016a), a simple case where after seeing the threshold binary decision rule h ( x ) = 2 · 1 [ x ≥ τ h ] − 1 , the agents will best response to it by maximizing the following utility function: u ( x, x (cid:48) )

Original: =
Revised: =

Original: h ( x (cid:48) ) − h
Revised: h ( x (cid:48) ) − h

Original: ( x ) − c ( x, x (cid:48) ) , where c ( x, x (cid:48) ) is the cost function for decision subjects to modify their feature from x to x (cid:48) .
Revised: ( x ) − c ( x, x (cid:48) ) , where c ( x, x (cid:48) ) is the cost function for decision subjects to modify their feature from x to x (cid:48) .

Original: We assume all agents are rational utility maximizers: they will only attempt to change their features when the beneﬁt of manipulation is greater than the cost (i.e. when c ( x, x (cid:48) ) ≤ 2 ) and agent will not change their feature if they are already accepted (i.e. h ( x ) = +1 ).
Revised: We assume all agents are rational utility maximizers: they will only attempt to change their features when the beneﬁt of manipulation is greater than the cost (i.e. when c ( x, x (cid:48) ) ≤ 2 ) and agent will not change their feature if they are already accepted (i.e. h ( x ) = +1 ).

Original: For a given threshold τ h and manipulation budget B , the theoretical best response of an agent with original feature x is: 
Revised: For a given threshold τ h and manipulation budget B , the theoretical best response of an agent with original feature x is: ∆( x ) = arg max x (cid:48) u

Original: [Equation]
Revised: ( x, x (cid:48) ) s.t. c ( x, x (cid:48) ) ≤ B.

Original: To make the problem tractable and meaningful, we further specify the following setups:
Revised: To make the problem tractable and meaningful, we further specify the following setups:

Original: Setup 1 .
Revised: Setup 1 .

Original: (Agent’s Initial Feature Distribution) Agents’ initial features are uniformly distributed between [0 , 1] ∈ R 1 .
Revised: (Initial Feature) Agents’ initial features are uniformly distributed between [0 , 1] ∈ R 1 .

Original: Setup 2 .
Revised: Setup 2 .

Original: (Agent’s Cost Function)
Revised: (Agent’s Cost Function)

Original: The cost of changing from x to x (cid:48) is proportional to the distance between them: c ( x, x (cid:48) )
Revised: The cost of changing from x to x (cid:48) is proportional to the distance between them: c ( x, x (cid:48) )

Original: = (cid:107) x − x (cid:48) (cid:107) .
Revised: = (cid:107) x − x (cid:48) (cid:107) .

Original: Setup 2 implies that only agents whose features are in between [ τ h − B, τ h ) will attempt to change their feature.
Revised: Setup 2 implies that only agents whose features are in between [ τ h − B, τ h ) will attempt to change their feature.

Original: We also assume that feature updates are probabilistic , such that agents with features closer to the decision boundary τ h have a greater chance of updating their feature and each updated feature x (cid:48) is sampled from a uniform distribution depending on τ h , B , and x (see Setup 3 & 4):
Revised: We also assume that feature updates are probabilistic , such that agents with features closer to the decision boundary τ h have a greater chance of updating their feature and each updated feature x (cid:48) is sampled from a uniform distribution depending on τ h , B , and x (see Setup 3 & 4):

Original: Setup 3 .
Revised: Setup 3 .

Original: (Agent’s Success Manipulation Probability)
Revised: (Agent’s Success Manipulation Probability)

Original: For agents who attempt to update their features, the probability of a successful feature update is P ( X (cid:48) (cid:54) =
Revised: For agents who attempt to update their features, the probability of a successful feature update is P ( X (cid:48) (cid:54) =

Original: X ) = 1 − | x − τ h | B .
Revised: X ) = 1 − | x − τ h | B .

Original: Setup 4 (Newly Adapted Feature’s Distribution) .
Revised: Setup 4 (Adapted Feature’s Distribution) .

Original: An agent’s updated feature x (cid:48) , given originalfeature x , manipulation budget B , and classiﬁcation boundary τ h , is sampled as X (cid:48) ∼ Unif ( τ h , τ h +  B − x  ) .
Revised: An agent’s updated feature x (cid:48) , given original x , manipulation budget B , and classiﬁcation boundary τ h , is sampled as X (cid:48) ∼ Unif ( τ h , τ h + | B − x | ) .

Original: Setup 4 aims to capture the fact that even though agent targets to change their feature to the decision boundary τ h (i.e. the least cost action to get a favorable prediction outcome), they might end up reaching to a feature that is beyond the decision boundary.
Revised: Setup 4 aims to capture the fact that even though agent targets to change their feature to the decision boundary τ h (i.e. the least cost action to get a favorable prediction outcome), they might end up reaching to a feature that is beyond the decision boundary.

Original: With the above setups, we can specify the bound in Theorem 4.2 for the strategic response setting as follows:
Revised: With the above setups, we can specify the bound in Theorem 4.2 for the strategic response setting as follows:

Original: Proposition 4.7 (Strategic ResponseSetting) .
Revised: Proposition [Equation] 7 (Strategic Response

Original: Err D ( h ∗ S ) ( h ∗ S ) − Err D ( h ∗ T ) ( h ∗ T ) ≤ (cid:113) 2 B 3 Err D S ( h ∗ T ) .
Revised: [Equation]

Original: We can see that the upper bound for strategic response depends on the manipulation budget B , and the error the ideal classiﬁer made on the source distribution Err D S ( h ∗ T ) .
Revised: We can see that the upper bound for strategic response depends on the manipulation budget B , and the error the ideal classiﬁer made on the source distribution Err D S ( h ∗ T ) .

Original: This aligns with our intuition that the smaller manipulation budget is, the less agents will change their features, thus leading to a tighter upper bound on the difference between Err h ∗ S ( h ∗ S ) and Err h ∗ T ( h ∗ T ) .
Revised: This aligns with our intuition that the smaller manipulation budget is, the less agents will change their features, thus leading to a tighter upper bound on the difference between Err h ∗ S ( h ∗ S ) and Err h ∗ T ( h ∗ T ) .

Original: This bound also allows us to bound this quantity even without the knowledge of the mapping between D ( h ) and h , since we can directly compute Err D S ( h ∗ T ) from the source distribution and an estimated optimal classiﬁer h ∗ T .
Revised: This bound also allows us to bound this quantity even without the knowledge of the mapping between D ( h ) and h , since we can directly compute Err D S ( h ∗ T ) from the source distribution and an estimated optimal classiﬁer h ∗ T .

Original:  T ARGET S HIFT
Revised:  T ARGET S HIFT

Original: We consider another popular domain adaptation setting known as target shift , in which the distribution of labels changes, but the distribution of features conditioned on the label remains the same:
Revised: We consider another popular domain adaptation setting known as target shift , in which the distribution of labels changes, but the distribution of features conditioned on the label remains the same:

Original: [Equation]
Revised: [Equation]

Original: For binary classiﬁcation, let ω ( h ) :
Revised: For binary classiﬁcation, let p ( h ) :

Original: = P D ( h ) ( Y = +1) , and P D ( h ) ( Y = − 1) = 1 − ω ( h ) .
Revised: = P D ( h ) ( Y = +1) , and P D ( h ) ( Y = − 1) = 1 − p ( h ) .

Original: Again, ω ( h ) encodes the induced adaptation from D S and h .
Revised: Again, p ( h ) encodes the induced adaptation from D S and h .

Original: Then we have for any proper loss function (cid:96) :
Revised: Then we have for any proper loss function (cid:96) :

Original: E D ( h ) [ (cid:96) ( h ; X, Y )]

Original: = ω ( h ) · E D ( h ) [ (cid:96) ( h ; X, Y )

Original: | Y = +1]

Original: + (1 − ω ( h )) · E D ( h ) [ (cid:96) ( h ; X, Y )

Original: | Y = − 1] = ω ( h ) · E D S

Original: [ (cid:96) ( h ; X, Y )

Original: | Y = +1]

Original: + (1 − ω ( h )) · E D S [ (cid:96) ( h ; X, Y ) | Y = − 1]
Revised: [Equation]

Original: We will adopt the following shorthands: Err + ( h ) :
Revised: We will adopt the following shorthands: Err + ( h ) :

Original: = E D S
Revised: = E D S

Original: [ (cid:96) ( h ; X, Y )
Revised: [ (cid:96) ( h ; X, Y )

Original: | Y
Revised: | Y

Original: =
Revised: =

Original: +1] , Err − ( h ) :=
Revised: +1] , Err − ( h ) :=

Original: E
Revised: E

Original: D S 
Revised: D S [

Original: [ (cid:96) ( h ; X, Y ) | Y = − 1] .
Revised: [Equation] ; X, Y ) | Y = − 1] .

Original: Note that Err + ( h ) , Err − ( h ) are both deﬁned on the conditional source distribution, which is invariant under the target shift assumption.
Revised: Note that Err + ( h ) , Err − ( h ) are both deﬁned on the conditional source distribution, which is invariant under the target shift assumption.

Original: 5. U PPER BOUND
Revised: 5. U PPER BOUND

Original: We again upper bound the transferability of h ∗ S under target shift.
Revised: We again upper bound the transferability of h ∗ S under target shift.

Original: Denote by D + the positive label distribution on D S ( P D S ( X = x | Y = +1) ) and D − the negative label distribution on D S ( P D S ( X = x | Y = − 1) ).
Revised: Denote by D + the positive label distribution on D S ( P D S ( X = x | Y = +1) ) and D − the negative label distribution on D S ( P D S ( X = x | Y = − 1) ).

Original: Let p :=
Revised: Let p :=

Original: P D S ( Y = +1) .
Revised: P D S ( Y = +1) .

Original: Theorem 5.1.
Revised: Theorem 5.1.

Original: For target shift, the difference between Err D ( h ∗ S ) ( h ∗ S ) and Err D ( h ∗ T ) ( h ∗ T ) bounds as:
Revised: For target shift, the difference between Err D ( h ∗ S ) ( h ∗ S ) and Err D ( h ∗ T ) ( h ∗ T ) bounds as:

Original: [Equation]
Revised: [Equation]

Original: The above upper bound consists of two components.
Revised: The above upper bound consists of two components.

Original: The ﬁrst quantity captures the difference between the two induced distributions D ( h ∗ S ) and D ( h ∗ T ) .
Revised: The ﬁrst quantity captures the difference between the two induced distributions D ( h ∗ S ) and D ( h ∗ T ) .

Original: The second quantity characterizes the difference between the two classiﬁers h ∗ S , h ∗ T on the source distribution.
Revised: The second quantity characterizes the difference between the two classiﬁers h ∗ S , h ∗ T on the source distribution.

Original: 5. L OWER B OUND
Revised: 5. L OWER B OUND

Original: Now we discuss lower bounds.
Revised: Now we discuss lower bounds.

Original: Denote by TPR S ( h ) and FPR S ( h ) the true positive and false positive rates of h on the source distribution D S .
Revised: Denote by TPR S ( h ) and FPR S ( h ) the true positive and false positive rates of h on the source distribution D S .

Original: We prove the following:
Revised: We prove the following:

Original: Theorem 5.2.
Revised: Theorem 5.2.

Original: For target shift, any model h must incur the following error on either D S or D ( h ) :
Revised: For target shift, any model h must incur the following error on either D S or D ( h ) :

Original: [Equation]
Revised: [Equation]

Original: S ( h ) | ). The proof extends the bound of Theorem 3.3 by further explicating each of d TV ( D Y | S , D Y ( h )) , d TV ( D h | S , D h ( h )) under the assumption of target shift.
Revised: The proof extends the bound of Theorem 3.3 by further explicating each of d TV ( D Y | S , D Y ( h )) , d [Equation] | S , D h ( h )) under the assumption of target shift.

Original: Since | TPR S ( h ) − FPR S ( h ) | < 0 unless we have a trivial classiﬁer that has either TPR S ( h ) = 1 , FPR S ( h ) = 0 or TPR S ( h ) = 0 , FPR S ( h ) = 1 , the lower bound is strictly positive.
Revised: Since | TPR S ( h ) − FPR S ( h ) | < 0 unless we have a trivial classiﬁer that has either TPR S ( h ) = 1 , FPR S ( h ) = 0 or TPR S ( h ) = 0 , FPR S ( h ) = 1 , the lower bound is strictly positive.

Original: Taking a closer look, the lower bound is determined linearly by how much the label distribution shifts: p − ω ( h ) .
Revised: Taking a closer look, the lower bound is determined linearly by how much the label distribution shifts: p − p ( h ) .

Original: The difference is further determined by the performance of h on the source distribution through 1 − | TPR S ( h ) − FPR S ( h ) | .
Revised: The difference is further determined by the performance of h on the source distribution through 1 − | TPR S ( h ) − FPR S ( h ) | .

Original: For instance, when 
Revised: For instance, when TPR

Original: TPR S ( h ) > FPR S ( h ) , the quality becomes FNR S ( h ) + FPR S ( h ) , that is the more error h makes, the larger the lower bound will be.
Revised: S ( h ) > FPR S ( h ) , the quality becomes FNR S ( h ) + FPR S ( h ) , that is the more error h makes, the larger the lower bound will be.

Original: 5. E XAMPLE U SING R EPLICATOR D YNAMICS
Revised: 5. E XAMPLE U SING R EPLICATOR D YNAMICS

Original: Let us instantiate the discussion using a speciﬁc ﬁtness function for the replicator dynamics model (Section 2.1), which is the prediction accuracy of h for class y :
Revised: Let us instantiate the discussion using a speciﬁc ﬁtness function for the replicator dynamics model (Section 2.1), which is the prediction accuracy of h for class y :

Original: [Equation]
Revised: [Equation]

Original: [Figure]

Original: Then we have E [ Fitness ( Y )]
Revised: Then we have E [ Fitness ( Y )]

Original: = 1 − Err D S ( h ) , and
Revised: = 1 − Err D S ( h ) , and

Original: [Figure]
Revised: [Equation]

Original:  the result back to our Theorem 5.1 we have
Revised: Plugging the result back to our Theorem 5.1 we have

Revised: [Figure]

Revised: 5 K 10 −10 −

Revised: V a l u e

Revised: MaxLB     5 K 10 −10 −

Revised: V a l u e

Original:  Proposition 5.3.
Revised: DiﬀUB Proposition 5.3.

Original: Under the replicator dynamics model in Eqn.
Revised: Under the replicator dynamics model in Eqn.

Original:  | ω ( h ∗ S ) − ω
Revised: (21), | ω ( h ∗ S ) − ω

Original: ( h ∗ T ) | bounds as:
Revised: ( h ∗ T ) | bounds as:

Original: [Equation]
Revised: [Equation]

Original:  That is, the difference between Err D ( h ∗ S ) ( h ∗ S ) and Err D ( h ∗ T ) ( h ∗ T ) is further dependent on the difference between the two classiﬁers’ performances on the source data D S .
Revised: [Figure] That is, the difference between Err D ( h ∗ S ) ( h ∗ S ) and Err D ( h ∗ T ) ( h ∗ T ) is further dependent on the difference between the two classiﬁers’ performances on the source data D S .

Original: This offers an opportunity to evaluate the possible error transferability using the source data only.
Revised: This offers an opportunity to evaluate the possible error transferability using the source data only.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We perform synthetic experiments using real-world data to demonstrate our bounds.
Revised: We perform synthetic experiments using real-world data to demonstrate our bounds.

Original: In particular, we use the FICO credit score data set (Board of Governors of the Federal Reserve System (US), 2007) which contains more than 300k records of TransUnion credit score of clients from different demographic groups.
Revised: In particular, we use the FICO credit score data set (Board of Governors of the Federal Reserve System (US), 2007) which contains more than 300k records of TransUnion credit score of clients from different demographic groups.

Original: For our experiment on the preprocessed FICO data set (Hardt et al., 2016b), we convert the cumulative distribution function (CDF) of TransRisk score among different groups into group-wise credit score densities, from which we generate a balanced sample to represent a population where groups have equal representations.
Revised: For our experiment on the preprocessed FICO data set (Hardt et al., 2016b), we convert the cumulative distribution function (CDF) of TransRisk score among different groups into group-wise credit score densities, from which we generate a balanced sample to represent a population where groups have equal representations.

Original: We demonstrate the application of our results in a series of resource allocations.
Revised: We demonstrate the application of our results in a series of resource allocations.

Original: We consider the hypothesis class of threshold classiﬁers and treat the classiﬁcation outcome as the decision received by individuals.
Revised: We consider the hypothesis class of threshold classiﬁers and treat the classiﬁcation outcome as the decision received by individuals.

Original: For each time step K =
Revised: For each time step K =

Original: k , we compute h ∗ S , the statistical optimal classiﬁer on the source distribution (i.e., the current reality for step K = k ), and update the credit score for each individual according to the received decision as the new reality for time
Revised: k , we compute h ∗ S , the statistical optimal classiﬁer on the source distribution (i.e., the current reality for step K = k ), and update the credit score for each individual according to the received decision as the new reality for time

Original: step K = k
Revised: step K = k

Original: + 1 .
Revised: + 1 .

Original: Details of the data generation is again deferred to Appendix C. We report our results in Figure 3.
Revised: Details of the data generation is again deferred to Appendix C. We report our results in Figure 3.

Original: We do observe positive gaps
Revised: We do observe positive gaps

Original: Err D ( h ∗ S ) ( h ∗ S ) − Err D
Revised: Err D ( h ∗ S ) ( h ∗ S ) − Err D

Original: ( h ∗ T ) ( h ∗ T ) , indicating the suboptimality of training on D S .
Revised: ( h ∗ T ) ( h ∗ T ) , indicating the suboptimality of training on D S .

Original: The gaps are well bounded by the theoretical upper bound (UB).
Revised: The gaps are well bounded by the theoretical upper bound (UB).

Original: Our lower bounds (LB) do return meaningful positive gaps, demonstrating the trade-offs that a classiﬁer has to suffer on either the source distribution or the induced target distribution.
Revised: Our lower bounds (LB) do return meaningful positive gaps, demonstrating the trade-offs that a classiﬁer has to suffer on either the source distribution or the induced target distribution.

Original:  We presented a sequence of model transferability results for settings where agents will respond to a deployed model.
Revised: Challenges in Minimizing Induced Risk and Concluding Remarks We presented a sequence of model transferability results for settings where agents will respond to a deployed model.

Original: The response leads to an induced distribution that the learner would not know before deploying the model.
Revised: The response leads to an induced distribution that the learner would not know before deploying the model.

Original: Our results cover for both a general response setting and for speciﬁc ones (covariate shift and target shift).
Revised: Our results cover for both a general response setting and for speciﬁc ones (covariate shift and target shift).

Revised: Looking forward to solving the induced risk minimization, the literature of domain adaptation has provided us solutions to minimize the risk on the target distribution via a nicely developed set of results (Sugiyama et al., 2008; 2007; Shimodaira, 2000).

Revised: This allows us to extend the solutions to minimize the induced risk too.

Revised: Nonetheless we will highlight additional computational challenges.

Revised: Let’s use the covariate shift setting.

Revised: The scenario for target shift is similar.

Original: E THICAL STATEMENT
Revised: For covariate shift, recall that earlier we derived the following fact: [Equation] This formula informs us that a promising solution that uses ω x ( h ) to perform reweighted ERM. There are two primary challenges when carrying out optimization of the above objective.

Revised: Of course, the primary challenge that stands in the way is how do we know ω x ( h ) .

Revised: When one could build models to predict the response D ( h ) and then ω x ( h ) (e.g., using the replicator dynamics model as we introduced earlier), one could rework the above loss and apply standard gradient descent approaches.

Revised: We provide a concrete example and discussion in Appendix E. Without making any assumptions on the mapping between h and D ( h ) , one can only potentially rely on the bandit feedbacks from the decision subjects to estimate the inﬂuence of h on D ( h ) - we also laid out a possibility in Appendix E too.

Revised: It can also be inferred from Eqn.

Revised: (6) that the second challenge is the induced risk minimization might not even be convex - due to the limit of space, we defer the detailed discussion again to the Appendix D.

Revised:  E THICAL STATEMENT

Original: The primary goal of our study is to put human in the center when considering domain shift.
Revised: The primary goal of our study is to put human in the center when considering domain shift.

Original: The development of the paper is fully aware of any fairness concerns and we expect positive societal impact.
Revised: The development of the paper is fully aware of any fairness concerns and we expect positive societal impact.

Original: Unawareness of the potential distribution shift might lead to unintended consequence when training a machine learning model.
Revised: Unawareness of the potential distribution shift might lead to unintended consequence when training a machine learning model.

Original: One goal of this paper is to raise awareness of this issue for a safe deployment of machine learning methods in high-stake societal applications.
Revised: One goal of this paper is to raise awareness of this issue for a safe deployment of machine learning methods in high-stake societal applications.

Original: A subset of our results are developed under assumptions (e.g., Theorem 4.6).
Revised: A subset of our results are developed under assumptions (e.g., Theorem 4.6).

Original: Therefore we want to caution readers of potential misinterpretation of applicability of the reported theoretical guarantees.
Revised: Therefore we want to caution readers of potential misinterpretation of applicability of the reported theoretical guarantees.

Original: Our contributions are mostly theoretical and our experiments use synthetic agent models to simulate distribution shift.
Revised: Our contributions are mostly theoretical and our experiments use synthetic agent models to simulate distribution shift.

Original: A future direction is to collect real human experiment data to support the ﬁndings.
Revised: A future direction is to collect real human experiment data to support the ﬁndings.

Original: Our paper ends with discussing the challenges in learning under the responding distribution and other objectives that might arise.
Revised: Our paper ends with discussing the challenges in learning under the responding distribution and other objectives that might arise.

Original: We believe this is a promising research direction for the machine learning community, both as a unaddressed technical problem and a stepstone for putting human in the center when training a machine learning model.
Revised: We believe this is a promising research direction for the machine learning community, both as a unaddressed technical problem and a stepstone for putting human in the center when training a machine learning model.

Original:  R EPRODUCIBILITY STATEMENT
Revised:  R EPRODUCIBILITY STATEMENT

Original: We provide the following checklist for the purpose of reproducibility:
Revised: We provide the following checklist for the purpose of reproducibility:

Original: Generals: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: Generals: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] We have stated our assumptions and limitations of the results.
Revised: [Yes] We have stated our assumptions and limitations of the results.

Original: We also discussed the limitations in the conclusion.
Revised: We also discussed the limitations in the conclusion.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] One of our work’s goals is to raise awareness of this issue for a safe deployment of machine learning methods in high-stake societal applications.
Revised: [Yes] One of our work’s goals is to raise awareness of this issue for a safe deployment of machine learning methods in high-stake societal applications.

Original: We discuss the potential misinterpretation of our results in conclusion.
Revised: We discuss the potential misinterpretation of our results in conclusion.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] We present the complete proofs in the appendix.
Revised: [Yes] We present the complete proofs in the appendix.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments .. (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We included experiment details in the appendix and submitted the implementation in the supplementary materials.
Revised: [Yes] We included experiment details in the appendix and submitted the implementation in the supplementary materials.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [N/A]
Revised: [N/A]

Original: In our controlled experiment, we do not tune parameters and do not observe a signiﬁcant variance in the results.
Revised: In our controlled experiment, we do not tune parameters and do not observe a signiﬁcant variance in the results.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A A PPENDIX
Revised: A A PPENDIX

Original: We arrange the appendix as follows:
Revised: We arrange the appendix as follows:

Original: • Appendix A.1 provides some real life scenarios where transparent models are useful or required.
Revised: • Appendix A.1 provides some real life scenarios where transparent models are useful or required.

Original: • Appendix A.2 provides comparisons of our setting and other sub-areas in domain adaptation.
Revised: • Appendix A.2 provides comparisons of our setting and other sub-areas in domain adaptation.

Original: • Appendix A.3 provides proof for Theorem 3.1.
Revised: • Appendix A.3 provides proof for Theorem 3.1.

Original: • Appendix A.4 provides proof for Theorem 3.2.
Revised: • Appendix A.4 provides proof for Theorem 3.2.

Original: • Appendix A.5 provides proof of Theorem 3.3.
Revised: • Appendix A.5 provides proof of Theorem 3.3.

Original: • Appendix A.6 provides proof for Proposition 4.1.
Revised: • Appendix A.6 provides proof for Proposition 4.1.

Original: • Appendix A.7 provides proof for Theorem 4.2.
Revised: • Appendix A.7 provides proof for Theorem 4.2.

Original: • Appendix A.8 provides proof for Theorem 4.6.
Revised: • Appendix A.8 provides proof for Theorem 4.6.

Original: • Appendix A.9 provides omitted assumptions and proof for Section 4.3.
Revised: • Appendix A.9 provides omitted assumptions and proof for Section 4.3.

Original: • Appendix A.10 provides proof for Theorem 5.1.
Revised: • Appendix A.10 provides proof for Theorem 5.1.

Original: • Appendix A.11 provides proof for Theorem B.1.
Revised: • Appendix A.11 provides proof for Theorem B.1.

Original: • Appendix A.12 provides proof for Proposition B.2.
Revised: • Appendix A.12 provides proof for Proposition B.2.

Original: • Appendix B provides additional lower bound and examples for the target shift setting.
Revised: • Appendix B provides additional lower bound and examples for the target shift setting.

Original: • Appendix C provides missing experimental details.
Revised: • Appendix C provides missing experimental results , including new experimental results using synthetic datasets generated according to causal graphs deﬁned in Figure 2. We also add additional experimental results on credit score data set.

Original: • Appendix D discusses challenges in minimizing induced risk.
Revised: • Appendix D discusses challenges in minimizing induced risk.

Revised: • Appendix E provides discussions on how to directly minimize the induced risk.

Revised: • Appendix F provides discussions on adding regularization to the objective function.

Revised: • Appendix G provides discussions on the tightness of our theoretical bounds.

Original: A. E XAMPLE U SAGES OF T RANSPARENT M ODELS
Revised: A. E XAMPLE U SAGES OF T RANSPARENT M ODELS

Original: As we mentioned in Section 1, there is an increasing requirement of making the decision rule to be transparent due to its potential consequences impacts to individual decision subject.
Revised: As we mentioned in Section 1, there is an increasing requirement of making the decision rule to be transparent due to its potential consequences impacts to individual decision subject.

Original: Here we provide the following reasons for using transparent models:
Revised: Here we provide the following reasons for using transparent models:

Original: • Government regulation may require the model to be transparent, especially in public services; •
Revised: • Government regulation may require the model to be transparent, especially in public services; •

Original: In some cases, companies may want to disclose their models so users will have explanations and are incentivized to better use the provided services.
Revised: In some cases, companies may want to disclose their models so users will have explanations and are incentivized to better use the provided services.

Original: • Regardless of whether models are published voluntarily, model parameters can often be inferred via well-known query “attacks”.
Revised: • Regardless of whether models are published voluntarily, model parameters can often be inferred via well-known query “attacks”.

Original: In addition, we name some concrete examples of some real-life applications:
Revised: In addition, we name some concrete examples of some real-life applications:

Original: • Consider the Medicaid health insurance program in the United States, which serves lowincome people.
Revised: • Consider the Medicaid health insurance program in the United States, which serves lowincome people.

Original: There is an obligation to provide transparency/disclose the rules (model to automate the decisions) that decide whether individuals qualify for the program — in fact, most public services have ”terms” that are usually set in stone and explained in the documentation.
Revised: There is an obligation to provide transparency/disclose the rules (model to automate the decisions) that decide whether individuals qualify for the program — in fact, most public services have ”terms” that are usually set in stone and explained in the documentation.

Original: Agents can observe the rules and will adapt their proﬁles to be qualiﬁed if needed.
Revised: Agents can observe the rules and will adapt their proﬁles to be qualiﬁed if needed.

Original: For instance, an agent can decide to provide additional documentation they need to guarantee approval.
Revised: For instance, an agent can decide to provide additional documentation they need to guarantee approval.

Original: For more applications along these lines, please refer to this report 5 . • Credit score companies directly publish their criteria for assessing credit risk scores.
Revised: For more applications along these lines, please refer to this report 5 . • Credit score companies directly publish their criteria for assessing credit risk scores.

Original: In loan application settings, companies actually have the incentive to release criteria to incentivize agents to meet their qualiﬁcations and use their services.
Revised: In loan application settings, companies actually have the incentive to release criteria to incentivize agents to meet their qualiﬁcations and use their services.

Original: Furthermore, making decision models transparent will gain the trust of users.
Revised: Furthermore, making decision models transparent will gain the trust of users.

Original: • It is also known that it is possible to steal model parameters, if agents have incentives to do so 6 .
Revised: • It is also known that it is possible to steal model parameters, if agents have incentives to do so 6 .

Original: For instance, spammers frequently infer detection mechanisms by sending different email variants; they then adjust their spam content accordingly.
Revised: For instance, spammers frequently infer detection mechanisms by sending different email variants; they then adjust their spam content accordingly.

Original: A. C OMPARISON OF OUR SETTING AND S OME A REAS IN D OMAIN A DAPTATION
Revised: A. C OMPARISON OF OUR SETTING AND S OME A REAS IN D OMAIN A DAPTATION

Original: We compare our setting (We address it as IDA, representing “induced domain adaptation”) with the following areas:
Revised: We compare our setting (We address it as IDA, representing “induced domain adaptation”) with the following areas:

Original: • Adversarial attack Chakraborty et al.
Revised: • Adversarial attack Chakraborty et al.

Original: ; Papernot et al.
Revised: ; Papernot et al.

Original: ; Song et al.
Revised: ; Song et al.

Original: (2019): in adversarial attack, the true label Y stays the same for the attacked feature, while in IDA, we allow the true label to change as well.
Revised: (2019): in adversarial attack, the true label Y stays the same for the attacked feature, while in IDA, we allow the true label to change as well.

Original: One can think of adversarial attack as a speciﬁc form of IDA where the induced distribution has a speciﬁc target, that is to maximize the classiﬁer’s error by only perturbing/modifying.
Revised: One can think of adversarial attack as a speciﬁc form of IDA where the induced distribution has a speciﬁc target, that is to maximize the classiﬁer’s error by only perturbing/modifying.

Original: Our transferability bound does, however, provide insights for how standard training results transfer to the attack setting.
Revised: Our transferability bound does, however, provide insights for how standard training results transfer to the attack setting.

Original: • Domain generalization Wang et al.
Revised: • Domain generalization Wang et al.

Original: (2021b)
Revised: (2021b)

Original: ; Li et al.
Revised: ; Li et al.

Original: ; Muandet et al.
Revised: ; Muandet et al.

Original: (2013): the goal of domain generalization is to learn amore general model that can be generalized to any unseen distribution; On the contrary, our focus is to understand how the performance of a classiﬁer trained on the source distribution degrades when evaluated on the induced distribution (which depends on how the population of decision subjects responds); this degradation depends on the classiﬁer itself.
Revised: (2013): the goal of domain generalization is to learn a model that can be generalized to any unseen distribution; Similar to our setting, one of the biggest challenges in domain generalization also the lack of target distribution during training. The major difference, however, is that our focus is to understand how the performance of a classiﬁer trained on the source distribution degrades when evaluated on the induced distribution (which depends on how the population of decision subjects responds); this degradation depends on the classiﬁer itself.

Original: • Test-time adaptation Varsavsky et al.
Revised: • Test-time adaptation Varsavsky et al.

Original: ; Wang et al.
Revised: ; Wang et al.

Original: (2021a)
Revised: (2021a)

Original: ; Nado et al.
Revised: ; Nado et al.

Original: (2021): the issue of test-time adaptation falls into the classical domain adaptation setting where the adaptation is independent of the model being deployed.
Revised: (2021): the issue of test-time adaptation falls into the classical domain adaptation setting where the adaptation is independent of the model being deployed.

Original: Applying this technique to solve our problem requires accessing data (either unsupervised or supervised) drawn from D S ( h ) for each h being evaluated during different training epochs.
Revised: Applying this technique to solve our problem requires accessing data (either unsupervised or supervised) drawn from D S ( h ) for each h being evaluated during different training epochs.

Original: A. P ROOF OFT HEOREM 3.
Revised: A. P ROOF OF

Revised: T HEOREM 3.

Original: Proof.
Revised: Proof.

Original: We ﬁrst establish two lemmas that will be helpful for bounding the errors of a pair of classiﬁers.
Revised: We ﬁrst establish two lemmas that will be helpful for bounding the errors of a pair of classiﬁers.

Original: Both are standard results from the domain adaption literature Ben-David et al.
Revised: Both are standard results from the domain adaption literature Ben-David et al.

Original: Lemma A.1.
Revised: Lemma A.1.

Original: For any hypotheses h, h (cid:48) ∈ H and distributions D , D (cid:48) ,
Revised: For any hypotheses h, h (cid:48) ∈ H and distributions D , D (cid:48) ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Deﬁne the-cross prediction disagreement between two classiﬁers h, h (cid:48) on a distribution D as
Revised: Deﬁne the-cross prediction disagreement between two classiﬁers h, h (cid:48) on a distribution D as

Original: Err D ( h, h (cid:48) ) :
Revised: Err D ( h, h (cid:48) ) :

Original: = P D ( h ( X ) (cid:54) = h (cid:48)
Revised: = P D ( h ( X ) (cid:54) = h (cid:48)

Original: ( X )) .
Revised: ( X )) .

Original: By the deﬁnition of the H− divergence,
Revised: By the deﬁnition of the H− divergence,

Original: [Equation]
Revised: [Equation]

Original: Another helpful lemma for us is the well-known fact that the 0-1 error obeys the triangle inequality (see, e.g., Crammer et al.
Revised: Another helpful lemma for us is the well-known fact that the 0-1 error obeys the triangle inequality (see, e.g., Crammer et al.

Original: Lemma A.2.
Revised: Lemma A.2.

Original: For any distribution D over instances and any labeling functions f 1 , f 2 , and f 3 , we have Err D ( f 1 , f 2 )
Revised: For any distribution D over instances and any labeling functions f 1 , f 2 , and f 3 , we have Err D ( f 1 , f 2 )

Original: ≤ Err D ( f 1 , f 3 ) + Err D ( f 2 , f 3 ) .
Revised: ≤ Err D ( f 1 , f 3 ) + Err D ( f 2 , f 3 ) .

Original: Denote by ¯ h ∗ the ideal joint hypothesis , which minimizes the combined error:
Revised: Denote by ¯ h ∗ the ideal joint hypothesis , which minimizes the combined error:

Original: [Equation]
Revised: [Equation]

Original: We have:
Revised: We have:

Original: [Equation]
Revised: [Equation]

Original: ( ¯ h ∗ )

Original: + Err D ( h ) ( h, ¯ h ∗ ) (Lemma A.2)

Original: ≤ Err D ( h ) (¯ h ∗ )

Original: + Err D S ( h, ¯ h ∗ )

Original: + (cid:12)(cid:12)

Original: Err D ( h ) ( h, ¯ h ∗ ) − Err D S ( h, ¯ h ∗ ) (cid:12)(cid:12)

Original: ≤ Err D ( h ) (¯ h ∗ )

Original: + Err D S ( h )

Original: + Err D S (¯ h ∗ )

Original: + 1 2 d H×H

Original: ( D S , D ( h ))

Original: (Lemma A.1) =

Original: Err D S ( h ) + λ D S →D

Original: ( h ) + 1 2 d H×H

Original: ( D S , D ( h )) .

Original: (Deﬁnition of ¯ h ∗ )

Original: A. P ROOF OF T HEOREM 3. 
Revised: A. P ROOF OF T HEOREM 3.

Original: Proof.
Revised: Proof.

Original: Invoking Theorem 3.1, and replacing h with h ∗ T and S with D ( h ∗ T ) , we have
Revised: Invoking Theorem 3. [Equation] and replacing h with h ∗ T and S with D ( h ∗ T ) , we have

Original: [Equation]
Revised: [Equation]

Original: Now observe that
Revised: Now observe that

Original: [Equation]
Revised: [Equation]

Original:  Adding Err D ( h ) ( h ) to both sides and rearranging terms yields
Revised: A.1) [Equation] [Equation] Adding Err D ( h ) ( h ) to both sides and rearranging terms yields

Original: [Equation]
Revised: [Equation]

Original: Dividing both sides by 2 completes the proof
Revised: Dividing both sides by 2 completes the proof

Original: [Equation]
Revised: [Equation]

Original: A. P ROOF OF T HEOREM 3. 
Revised: A. P ROOF OF T HEOREM 3.

Original: Proof.
Revised: Proof.

Original: Using the triangle inequality of d TV 
Revised: Using the triangle inequality of d TV , we have

Original: [Equation]
Revised: [Equation]

Original: we have

Original: [Equation]
Revised: [Equation]

Original: and by the deﬁnition of d TV , the divergence term d TV ( D Y | S , D Y ( h )) becomes 
Revised: and by the deﬁnition of d TV , the divergence term d TV ( D Y | S , D Y ( h )) becomes [Equation] [Equation] [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Revised: (cid:12)(cid:12)(cid:12)(cid:12)

Revised: [Equation]

Revised: (cid:12)(cid:12)(cid:12)(cid:12)

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: ] (cid:12)(cid:12)(cid:12)(cid:12)

Revised: [Equation]

Revised: [Equation]

Original: Similarly, we have
Revised: Similarly, we have

Original: [Equation]
Revised: [Equation]

Original: As a result, we have
Revised: As a result, we have

Original: [Equation]
Revised: [Equation]

Original:  equation 9) which implies
Revised: (by equation 8) which implies

Original: [Equation]
Revised: [Equation]

Original: A. P ROOF OF P ROPOSITION 4. 
Revised: A. P ROOF OF P ROPOSITION 4.

Original: Proof.
Revised: Proof.

Revised: [Equation]

Revised: =

Original: [Equation]
Revised: [Equation]

Original: A. P ROOF OF T HEOREM 4. 
Revised: A. P ROOF OF T HEOREM 4.

Original: Proof.
Revised: Proof.

Original: We start from the error induced by h ∗ S .
Revised: We start from the error induced by h ∗ S .

Original: Let the average importance weight induced by h ∗ S be ¯ ω ( h ∗ S ) =
Revised: Let the average importance weight induced by h ∗ S be ¯ ω ( h ∗ S ) =

Original: E D S [ ω x ( h ∗ S )] ; we add and subtract this from the error:
Revised: E D S [ ω x ( h ∗ S )] ; we add and subtract this from the error:

Original: Err

Original: [Equation]
Revised: [Equation]

Original: In fact, ¯ ω ( h ∗ S )
Revised: In fact, ¯ ω ( h ∗ S )

Original: = 1 , since
Revised: = 1 , since

Original: [Equation]
Revised: [Equation]

Original: Now consider any other classiﬁer h .
Revised: Now consider any other classiﬁer h .

Original: We have
Revised: We have

Original: Err
Revised: Err

Original: [Equation]
Revised: [Equation]

Original: (by optimality of h ∗ S on D 
Revised: (by optimality of h ∗ S on D S ) =

Original: [Equation]
Revised: [Equation]

Original: [Equation] (multiply by ¯ ω 
Revised: (multiply by ¯ ω ( h ∗ S ) = 1 )

Original: [Equation]
Revised: [Equation]

Original: = 1  ) [Equation] (add and subtract ¯ ω ( [Equation]
Revised: 1 ( h ( x ) (cid:54) = y )] (add and subtract ¯ ω ( h ∗

Original: [Equation]
Revised: S ) )

Original: Moving the error terms to one side, we have
Revised: + E D S

Original: [Equation] ( h ∗ S )
Revised: [( ω x ( h ∗ S )

Original: − Err D
Revised: − ¯ ω

Original: ( h) ( h ) ≤ Cov ( ω x ( h ∗ S ) , 1 ( h ∗ S ( x ) (cid:54) = y ))
Revised: ( h ∗ S )) · 1 ( h ∗ S ( x ) (cid:54) = y )] =

Original: − Cov ( ω x ( h ), 1 ( h( x ) (cid:54) = y )) ≤ (cid:113) Var ( ω x ( h ∗ S )) Var ( 1 ( h ∗ S ( x ) (cid:54) = y ))
Revised: Err D ( h ) ( h ) + Cov ( ω x ( h ∗ S ) , 1 ( h ∗ S ( x ) (cid:54) = y ))

Original: ( | Cov ( X, Y ) | ≤ (cid:112) Var ( X ) · Var ( Y ) ) 
Revised: − Cov ( ω x ( h ) , 1 ( h ( x ) (cid:54) = y ))

Original: + (cid:112) Var ( ω x ( h )) · Var ( 1 ( h ( x ) (cid:54) = y )) = (cid:113) Var ( ω x ( h ∗ S )) · Err S ( h ∗ S )(1 − Err S ( h ∗ S ))
Revised: Moving the error terms to one side, we have

Original: + (cid:112) Var ( ω x ( h )) · Err D S ( h )(1 − Err D S ( h )) ≤ (cid:113) Var ( ω x ( h ∗ S )) · Err S ( h ∗ S ) + (cid:112) Var ( ω x ( h )) ·
Revised: [Equation]

Original: Err D S ( h ) ( 1 − Err D S ( h ) ≤ 1 ) ≤ (cid:112) Err D S ( h ) · (cid:18)(cid:113) Var ( ω x ( h ∗ S )) + (cid:112) Var ( ω x ( h))
Revised: Since this holds for any h

Original: (cid:19)
Revised: [Equation]

Original: Since this holds for any h , it certainly holds for h = h ∗ T .
Revised: it certainly holds for h = h ∗ T .

Original: A. O MITTED A SSUMPTIONS AND P ROOF OF T HEOREM 4. 
Revised: A. O MITTED A SSUMPTIONS AND P ROOF OF T HEOREM 4.

Original: Denote X
Revised: Denote X

Original: + ( h ) =
Revised: + ( h ) =

Original: { x : ω x ( h ) ≥ 1 } and X − ( h ) =
Revised: { x : ω x ( h ) ≥ 1 } and X − ( h ) =

Original: { x : ω x ( h ) < 1 } .
Revised: { x : ω x ( h ) < 1 } .

Original: First we observe that
Revised: First we observe that

Original: [Equation]
Revised: [Equation]

Original: This is simply because of (cid:82) x
Revised: This is simply because of (cid:82) x

Original: P D S( X = x ) ·
Revised: P D S

Original: [Equation]
Revised: [Equation]

Original:  x[Equation]
Revised: X = x ) · ω x

Original: [Equation]
Revised: [Equation]

Original: = (cid:82) x PD ( h ) (
Revised: P

Original: X = x ) dx = 1 .
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Notice that in the setting of binary classiﬁcation
Revised: Notice that in the setting of binary classiﬁcation

Original: [Equation]
Revised: [Equation]

Original: we can write the total variation distance between D Y | S and D Y ( h ) as the difference between the probability of Y
Revised: we can write the total variation distance between D Y | S and D Y ( h ) as the difference between the probability of Y

Original: = +

Revised: = +

Original: and the probability of Y = − 1 :
Revised: and the probability of Y = − 1 :

Original: [Equation]
Revised: [Equation]

Original: Similarly we have
Revised: Similarly we have

Original: [Equation]
Revised: [Equation]

Original: We can further expand the total variation distance between D Y | S and D Y ( h ) as follows:
Revised: We can further expand the total variation distance between D Y | S and D Y ( h ) as follows:

Original: [Equation]
Revised: [Equation]

Original: Similarly, by assumption 4.4 and equation equation
Revised: Similarly, by assumption 4.4 and equation equation

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Thus we can bound the difference between d TV ( D Y | S , D Y ( h )) and d TV ( D h | S , D h ( h )) as follows:
Revised: Thus we can bound the difference between d TV ( D Y | S , D Y ( h )) and d TV ( D h | S , D h ( h )) as follows:

Original: [Equation]
Revised: [Equation]

Original: Assumption 4.5)

Original: [Equation]
Revised: [Equation]

Original: Combining the above with Theorem
Revised: Combining the above with Theorem

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: A. O MITTED DETAILS FOR S ECTION 4. 
Revised: A. O MITTED DETAILS FOR S ECTION 4.

Original: With Setup 2 - Setup 4, we can further specify the important weight w x ( h ) for the strategic response setting:
Revised: With Setup 2 - Setup 4, we can further specify the important weight w x ( h ) for the strategic response setting:

Original: Lemma A.3.
Revised: Lemma A.3.

Original: Recall the deﬁnition for the covariate shift important weight coefﬁcient ω x ( h ) :
Revised: Recall the deﬁnition for the covariate shift important weight coefﬁcient ω x ( h ) :

Original: = P
Revised: = P

Original: D [Equation] h ) ( X = x ) P
Revised: D ( h ) ( X = x ) P

Original: DS ( X = x ) , for our strategic response setting, we have,
Revised: DS ( X = x ) , for our strategic response setting, we have,

Original: [Equation]
Revised: [Equation]

Original: Proof for Lemma A.3:
Revised: Proof for Lemma A.3:

Original: Proof.
Revised: Proof.

Original: We discuss the induced distribution D ( h ) by cases:
Revised: We discuss the induced distribution D ( h ) by cases:

Original: • For the features distributed between [0 , τ h − B ] : since we assume the agents are rational, under assumption 2, agents with feature that is smaller than [0 , τ h − B ] will not perform any kinds of adaptations, and no other agents will adapt their features to this range of features either, so the distribution between [0 , τ h − B ] will remain the same as before.
Revised: • For the features distributed between [0 , τ h − B ] : since we assume the agents are rational, under assumption 2, agents with feature that is smaller than [0 , τ h − B ] will not perform any kinds of adaptations, and no other agents will adapt their features to this range of features either, so the distribution between [0 , τ h − B ] will remain the same as before.

Original: • For the target distribution between [ τ h − B, τ h ] can be directly calculated from assumption 3.
Revised: • For the target distribution between [ τ h − B, τ h ] can be directly calculated from assumption 3.

Original: • For distribution between [ τ h , τ h + B ] , consider a particular feature x (cid:63) ∈ [ τ h , τ h + B ] , under Setup 4, we know its new distribution becomes:
Revised: • For distribution between [ τ h , τ h + B ] , consider a particular feature x (cid:63) ∈ [ τ h , τ h + B ] , under Setup 4, we know its new distribution becomes:

Original: [Equation]
Revised: [Equation]

Original: • For the target distribution between [ τ h + B, 1] : under assumption 2 and 4, we know that no agents will change their feature to this feature region.
Revised: • For the target distribution between [ τ h + B, 1] : under assumption 2 and 4, we know that no agents will change their feature to this feature region.

Original: So the distribution between [ τ h + B, 1] remains the same as the source distribution.
Revised: So the distribution between [ τ h + B, 1] remains the same as the source distribution.

Original: Recall the deﬁnition for the covariate shift important weight coefﬁcient ω x ( h ) :
Revised: Recall the deﬁnition for the covariate shift important weight coefﬁcient ω x ( h ) :

Original: = P D ( h ) ( X = x ) P DS ( X = x ) , the distribution of ω x ( h ) after agents’ strategic responding becomes:
Revised: = P D ( h ) ( X = x ) P DS ( X = x ) , the distribution of ω x ( h ) after agents’ strategic responding becomes:

Original: [Equation]
Revised: [Equation]

Original: Proof for Proposition 4.7:
Revised: Proof for Proposition 4.7:

Original: Proof.
Revised: Proof.

Original: According to Lemma A.3, we can compute the variance of w x ( h ) as Var ( w x ( h ))
Revised: According to Lemma A.3, we can compute the variance of w x ( h ) as Var ( w x ( h ))

Original: =
Revised: =

Original: E ( w x ( h ) 2 ) − E ( w x ( h ) 2 ) = 23 B .
Revised: E ( w x ( h ) 2 ) − E ( w x ( h ) 2 ) = 23 B .

Original: Then by plugging it to the general bound for Theorem 4.2 gives us the desirable result.
Revised: Then by plugging it to the general bound for Theorem 4.2 gives us the desirable result.

Original: A. P ROOF OF T HEOREM 5. 
Revised: A. P ROOF OF T HEOREM 5.

Original: Proof.
Revised: Proof.

Original: Deﬁning p := 
Revised: Deﬁning p := P

Original: P D S ( Y = +1) , we have

Original: [Equation]
Revised: [Equation]

Original: (by deﬁnitions of ω ( h∗ S )  , Err +
Revised: ( Y = +1) , p ( h ) = P D ( h ) ( Y = +1) , we have

Original: [Equation]
Revised: [Equation]

Original:  h
Revised: (by deﬁnitions of p ( h

Original: [Equation]
Revised: [Equation]

Original: ) ,and Err
Revised: , Err

Original: [Equation]
Revised: [Equation]

Original:  h [Equation] S [Equation]
Revised: , and Err − ( h ∗ S ) )

Original: [Equation]
Revised: [Equation]

Original: We can expand (I) as follows:
Revised: We can expand (I) as follows:

Original: [Equation]
Revised: [Equation]

Original: (by optimality of h
Revised: optimality of h

Original: [Equation]
Revised: [Equation]

Original: S on D S )
Revised: on D [Equation]

Original: [Equation]
Revised: [Equation]

Original: Plugging this back into equation 14, we have
Revised: Plugging this back into equation 13, we have

Original: [Equation]
Revised: [Equation]

Original: Notice that
Revised: Notice that

Original: [Equation]
Revised: [Equation]

Original: where D u is a distribution with uniform prior.
Revised: where D u is a distribution with uniform prior.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Adding together these two equations yields
Revised: Adding together these two equations yields

Original: [Equation]

Original: T ) − Err − ( h ∗ T )] =

Original: [Equation]

Original: [Equation]

Original: p ) · (

Original: [Equation]

Original: u (

Original: [Equation]

Original: ∗ S (

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: ( p

Original: [Equation]

Original: [Equation]

Original: T )) · (0 .

Original: [Equation]

Original: P

Original: [Equation]

Original: u (

Original: [Equation]

Original: ∗ T ( X ) = +1))

Original: =

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: u

Original: [Equation]

Original: S

Original: [Equation]

Original: h

Original: [Equation]
Revised: [Equation]

Original: u

Original: [Equation]

Original: [Equation]

Original: X

Original: [Equation]

Original: )) ≤

Original: [Equation]

Original: D u

Original: [Equation]

Original: 1) | )

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: D u

Original: [Equation]

Original: ∗

Original: [Equation]

Original: X )

Original: [Equation]

Original: 1) | (15)

Original: Meanwhile,|
Revised: Meanwhile,

Original: [Equation]
Revised: [Equation]

Original: X

Original: [Equation]

Original: |

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: |

Original: [Equation]

Original: T

Original: [Equation]

Original: X )

Original: [Equation]

Original: 1) |

Original: [Equation]

Original: [Equation]

Original: · | P D| Y =

Original: [Equation]

Original: 1 ( h ∗ S ( X ) = +1)

Original: [Equation]

Original: P D| Y = − 1 ( h ∗ T ( X )

Original: [Equation]

Original: +1)

Original: |

Original: [Equation]

Original: [Equation]

Original: ∗

Original: [Equation]

Original: [Equation]

Original: ∗ T ))

Original: [Equation] ∗ [Equation] Combining equation 15 and equation 16 gives
Revised: Combining equation 14 and equation 15 gives

Original: [Equation]
Revised: [Equation]

Original: A. P ROOF OF T HEOREM B. 
Revised: A. P ROOF OF T HEOREM B.

Original: We will make use of the following fact:
Revised: We will make use of the following fact:

Original: Lemma A.4.
Revised: Lemma A.4.

Original: Under label shift, TPR S ( h ) =
Revised: Under label shift, TPR S ( h ) =

Original: TPR h ( h ) and FPR S ( h ) =
Revised: TPR h ( h ) and FPR S ( h ) =

Original: FPR h ( h ) .
Revised: FPR h ( h ) .

Original: Proof.
Revised: Proof.

Original: We have
Revised: We have

Original: [Equation]

Original: (by deﬁnition of label shift)

Original: [Equation]
Revised: [Equation]

Original: The argument for TPR h ( h ) =
Revised: The argument for TPR h ( h ) =

Original: TPR S ( h ) is analogous.
Revised: TPR S ( h ) is analogous.

Original: Now we proceed to prove the theorem
Revised: Now we proceed to prove the theorem.

Original: [Equation]

Original: Proof of Theorem B.1.
Revised: Proof of Theorem B.1.

Original: In section 3.2 we showed a general lower bound on the maximum of Err D S ( h ) and Err D ( h ) ( h ) :
Revised: In section 3.2 we showed a general lower bound on the maximum of Err D S ( h ) and Err D ( h ) ( h ) :

Original: [Equation]
Revised: [Equation]

Original: In the case of label shift [Equation] and by the deﬁnitions of p and ω ( h ) ,
Revised: In the case of label shift, and by the deﬁnitions of p and p ( h ) ,

Original: [Equation]
Revised: [Equation]

Original: In addition
Revised: In addition

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Similarly
Revised: Similarly

Original: [Equation]
Revised: [Equation]

Original: (by Lemma A.4) (19)
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]

Original: (By equation 19 and equation 18)

Original: [Equation]
Revised: [Equation]

Original:  which yields:
Revised: 19) which yields:

Original: [Equation]
Revised: [Equation]

Original: By equation 17 and equation 20) completing the proof.
Revised: equation 19) completing the proof [Equation]

Original: A. P ROOF OF P ROPOSITION B. 
Revised: A. P ROOF OF P ROPOSITION B.

Original: Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: The inequality above is due to Lemma 7 of Liu & Liu (2015).
Revised: The inequality above is due to Lemma 7 of Liu & Liu (2015).

Original: B L OWER B OUND AND E XAMPLE FOR T ARGET S HIFT B. L OWER B OUND
Revised: B L OWER B OUND AND E XAMPLE FOR T ARGET S HIFT B. L OWER B OUND

Original: Now we discuss lower bounds.
Revised: Now we discuss lower bounds.

Original: Denote by TPR S ( h ) and FPR S ( h ) the true positive and false positive rates of h on the source distribution D S .
Revised: Denote by TPR S ( h ) and FPR S ( h ) the true positive and false positive rates of h on the source distribution D S .

Original: We prove the following:
Revised: We prove the following:

Original: Theorem B.1.
Revised: Theorem B.1.

Original: Under target shift, any model h must incur the following error on either the D S or
Revised: Under target shift, any model h must incur the following error on either the D S or

Original: D ( h ) :
Revised: D ( h ) :

Original: [Equation]
Revised: [Equation]

Original: The proof extends the bound of Theorem 3.3 by further explicating each of d TV ( D Y | S , D Y ( h )) , d TV ( D h | S , and D h ( h )) under the assumption of target shift.
Revised: The proof extends the bound of Theorem 3.3 by further explicating each of d TV ( D Y | S , D Y ( h )) , d TV ( D h | S , and D h ( h )) under the assumption of target shift.

Original: Since | TPR S ( h ) − FPR S ( h ) | < 0 unless we have a trivial classiﬁer that has either TPR S ( h ) = 1 , FPR S ( h ) = 0 or TPR S ( h ) = 0 , FPR S ( h ) = 1 , the lower bound is strictly positive.
Revised: Since | TPR S ( h ) − FPR S ( h ) | < 0 unless we have a trivial classiﬁer that has either TPR S ( h ) = 1 , FPR S ( h ) = 0 or TPR S ( h ) = 0 , FPR S ( h ) = 1 , the lower bound is strictly positive.

Original: Taking a closer look, the lower bound is determined linearly by how much the label distribution shifts: p − ω ( h ) .
Revised: Taking a closer look, the lower bound is determined linearly by how much the label distribution shifts: p − p ( h ) .

Original: The difference is further determined by the performance of h on the source distribution through 1 − | TPR S ( h ) − FPR S ( h ) | .
Revised: The difference is further determined by the performance of h on the source distribution through 1 − | TPR S ( h ) − FPR S ( h ) | .

Original: For instance, when 
Revised: For instance, when TPR

Original: TPR S ( h ) > FPR S ( h ) , the quality becomes FNR S ( h ) + FPR S ( h ) , that is the more error h makes, the larger the lower bound will be.
Revised: S ( h ) > FPR S ( h ) , the quality becomes FNR S ( h ) + FPR S ( h ) , that is the more error h makes, the larger the lower bound will be.

Original: B. E XAMPLE U SING R EPLICATOR D YNAMICS
Revised: B. E XAMPLE U SING R EPLICATOR D YNAMICS

Original: Let us instantiate the discussion using a speciﬁc ﬁtness function for the replicator dynamics model (Section 2.1), which is the prediction accuracy of h for class +1 :
Revised: Let us instantiate the discussion using a speciﬁc ﬁtness function for the replicator dynamics model (Section 2.1), which is the prediction accuracy of h for class +1 :

Original: [Equation]
Revised: [Equation]

Original: Then we have E
Revised: Then we have E

Original: [ Fitness of Y ] =
Revised: [ Fitness of Y ] =

Original: Err D S ( h ) , and 
Revised: Err D S ( h ) , and p

Original: [Equation]
Revised: [Equation]

Original: Plugging the result back to our Theorem 5.1 we have
Revised: Plugging the result back to our Theorem 5.1 we have

Original: Proposition B.2.
Revised: Proposition B.2.

Original: Under the replicator dynamics model in Eqn.
Revised: Under the replicator dynamics model in Eqn.

Original:  | ω ( h ∗ S ) − ω ( h ∗ T ) | further bounds as:
Revised: (21), | p ( h ∗ S ) − p ( h ∗ T ) | further bounds as:

Original: [Equation]
Revised: [Equation]

Original: That is, the difference between Err D ( h ∗ S ) ( h ∗ S ) and Err D ( h ∗ T ) ( h ∗ T ) is further dependent on the difference between the two classiﬁers’ performances on the source data D S .
Revised: That is, the difference between Err D ( h ∗ S ) ( h ∗ S ) and Err D ( h ∗ T ) ( h ∗ T ) is further dependent on the difference between the two classiﬁers’ performances on the source data D S .

Original: This offers an opportunity to evaluate the possible error transferability using the source data only.
Revised: This offers an opportunity to evaluate the possible error transferability using the source data only.

Original: C M ISSING E XPERIMENTAL D ETAILS 
Revised: C M ISSING E XPERIMENTAL D ETAILS C. S YNTHETIC E XPERIMENTS U SING DAG

Revised: Synthetic experiments using simulated data We generate synthetic data sets from structural equation models described on simple causal DAG in Figure 2 for covariate shift and target shift.

Revised: To generate the induced distribution D ( h ) , we posit a speciﬁc adaptation function ∆ : R d × H → R d , so that when an input x encounters classiﬁer h ∈ H , its induced features are precisely x (cid:48) =

Revised: ∆( x, h ) .

Revised: We provide details of the data generation processes and adaptation functions in Appendix C.

Original: On the preprocessed FICO credit score data set Board of Governors of the Federal Reserve System (US) (2007); Hardt et al.
Revised: We take our training data set { x 1 , . . . , x n } and learn a “base” logistic regression model h ( x ) = σ ( w · x ) 7 . We then consider the hypothesis class H :

Revised: =

Revised: { h τ | τ ∈ [0 , 1] } , where h τ ( x ) := 2 · 1 [ σ ( w · x ) > τ ] − 1 .

Revised: To compute h ∗ S , the model that performs best on the source distribution, we simply vary τ and take the h τ with lowest prediction error.

Revised: Then, we posit a speciﬁc adaptation function ∆( x, h τ ) .

Revised: Finally, to compute h ∗ T , we vary τ from 0 to 1 and ﬁnd the classiﬁer h τ that minimizes the prediction error on its induced data set { ∆( x 1 , h τ ) , . . .

Revised: , ∆( x n , h τ ) } .

Revised: We report our results in Figure 4.

Revised: Covariate Shift We specify the causal DAG for covariate shift setting in the following way:

Revised: [Equation]

Revised: where σ 2 2 and σ 2 3 are parameters of our choices.

Revised: Adaptation function We assume the new distribution of feature X (cid:48) 1 will be generated in the following way:

Revised: [Equation]

Revised: where c ∈ R 1 > 0 is the parameter controlling how much the prediction h ( X ) affect the generating of X (cid:48) 1 , namely the magnitude of distribution shift.

Revised: Intuitively, this adaptation function means that if a feature x is predicted to be positive ( h ( x ) = +1 ), then decision subjects are more likely to adapt to that feature in the induced distribution; Otherwise, decision subjects are more likely to be moving away from x since they know it will lead to a negative prediction.

Revised: Target Shift We specify the causal DAG for target shift setting in the following way:

Revised: [Equation]

Original: (2016b), we convert the cumulative distribution function (CDF) of TransRisk score among demographic groups (denoted as A , including Black, Asian, Hispanic, and White) into group-dependent densities of  the credit score.
Revised: where N [0 , 1] represents a truncated Gaussian distribution taken value between 0 and 1. α , µ y , σ 2 , σand σ 23 are parameters of our choices. Adaptation function We assume the new distribution of the qualiﬁcation Y (cid:48) will be updated in the following way: [Equation]

Revised: where { h, y

Revised: [Equation]

Revised: ∈ {

Revised: [Equation]

Revised: where 0 ≤ c hy ∈ R 1 ≤ 1 represents the likelihood for a person with original qualiﬁcation Y =

Revised: y and get predicted as h ( X )

Revised: = h to be qualiﬁed in the next step ( Y (cid:48)

Revised: = +1 ).

Revised: Discussion of the Results For all four datasets, we do observe positive gaps Err D ( h ∗ S )

Revised: ( h ∗ S ) −

Revised: Err D ( h ∗ T ) ( h ∗ T ) , indicating the suboptimality of training on D S .

Revised: The gaps are well bounded by the theoretical results.

Revised: For lower bound, the empirical observation and the theoretical bounds are roughly within the same magnitude except for one target shift dataset, indicating the effectiveness of our theoretical result.

Revised: For upper bound, for target shift, the empirical observations are well within the same magnitude of the theoretical bounds while the results for the covariate shift are relatively loose.

Revised: C. S YNTHETIC E XPERIMENTS U SING R EAL - WORLD D ATA

Revised: On the preprocessed FICO credit score data set (Board of Governors of the Federal Reserve System (US), 2007; Hardt et al., 2016b), we convert the cumulative distribution function (CDF) of TransRisk score among demographic groups (denoted as A , including Black, Asian, Hispanic, and White) into group-dependent densities of the credit score.

Original: We then generate a balanced sample where each group has equal representation, with credit scores (denoted as Q ) initialized by sampling from the corresponding group-dependent density.
Revised: We then generate a balanced sample where each group has equal representation, with credit scores (denoted as Q ) initialized by sampling from the corresponding group-dependent density.

Original: The value of attributes for each data point is then updated under a speciﬁed dynamics (as detailed below) to model the real-world scenario of repeated resource allocation (with decision denoted as D ). 
Revised: The value of attributes for each data point is then updated under a speciﬁed dynamics (detailed in Appendix C.2.1) to model the real-world scenario of repeated resource allocation (with decision denoted as D ). C.2. P ARAMETERS FOR D YNAMICS

Original: Since we are considering the dynamic setting, we further specify the data generating process in the following way (from time step T = t to T = t + 1 ):
Revised: Since we are considering the dynamic setting, we further specify the data generating process in the following way (from time step T = t to T = t + 1 ):

Original: [Equation]
Revised: [Equation]

Original: Bernoulli( q t ) for a given value of Q t = qt
Revised: ( q t ) for a given value of [Equation] q

Original: [Equation]
Revised: [Equation]

Original: ﬁxed population) where {·} (0 , 1] represents truncated value between the interval (0 , 1] , f t ( · ) represents the decision policy from input features, and (cid:15) 1 , (cid:15) 2 , σ are parameters ofour choices. {+In our experiments, we set (cid:15) 
 = (cid:15) 2 = σ = 0 .+}
Revised: where {·} (0 , 1] represents truncated value between the interval (0 , 1] , f t ( · ) represents the decision policy from input features, and (cid:15) 1 , (cid:15) 2 , σ are parameters of choices. {+In our experiments, we set (cid:15) 
 = (cid:15) 2 = σ = 0 .+}

Original: Within the same time step, i.e., for variables that share the subscript t , Q t and A t are root causes for all other variables ( X t, 1 , X t, 2 , X t, 3 , D t , Y t ).
Revised: Within the same time step, i.e., for variables that share the subscript t , Q t and A t are root causes for all other variables ( X t, 1 , X t, 2 , X t, 3 , D t , Y t ).

Revised: At each time step T = t , the institution ﬁrst estimates the credit score Q t (which is not directly visible to the institution, but is reﬂected in the visible outcome label Y t ) based on ( A t , X t, 1 , X t, 2 , X t, 3 ) , then produces the binary decision D t according to the optimal threshold (in terms of the accuracy).

Original: For different time steps, e.g., from T = t to T = t + 1 , the new distribution at T =t
Revised: For different time steps, e.g., from T = t to T = t +1 , the new distribution at T =

Original: + 1 is induced by the deployment of the decision policy D t .
Revised: t +1 is induced by the deployment of the decision policy D t .

Original: Such impact is modeled by a multiplicative update in Q t +1 from Q t with parameters (or functions)
Revised: Such impact is modeled by a multiplicative update in Q t + from Q t with parameters (or functions)

Original: α D ( · ) and α Y ( · ) that depend on D t and Y t , respectively. 
Revised: α D ( · ) and α Y ( · ) that depend on D t and Y t , respectively. In our experiments, we set α D = 0 . 01 and α Y = 0 .

Original: D C HALLENGES IN M INIMIZING I NDUCED R ISK D. C OMPUTATIONAL C HALLENGES
Revised: 005 to capture the scenario where one-step inﬂuence of the decision on the credit score is stronger than that for ground truth label. [Figure] C.2. A DDITIONAL E XPERIMENTAL R ESULTS In this section we present additional experimental results on the real-world FICO credit score data set.

Revised: With the initialization of the distribution of credit score Q and the speciﬁed dynamics, we present results comparing the inﬂuence of vanilla regularization terms in decision-making (when estimating the credit score Q ) on the calculation of bounds for induced risks.

Revised: 8 In particular, we consider Lnorm (Figure 5) and L 2 norm (Figure 6) regularization terms when optimizing decision-making policies on the source domain.

Revised: As we can see from the results, applying vanilla regularization terms (e.g., L 1 norm and L 2 norm) on source domain without speciﬁc considerations of the inducing-risk mechanism does not provide signiﬁcant performance improvement in terms of smaller induced risk.

Revised: For example, there is no signiﬁcant decrease of the term Diff as the regularization strength increases, for both L 1 norm (Figure 5) and L 2 norm (Figure 6) regularization terms.

Revised: [Figure]

Revised: D C HALLENGES IN M INIMIZING I NDUCED R ISK D. C OMPUTATIONAL C HALLENGES

Original: The literature of domain adaptation has provided us solutions to minimize the risk on the target distribution via a nicely developed set of results Sugiyama et al.
Revised: The literature of domain adaptation has provided us solutions to minimize the risk on the target distribution via a nicely developed set of results Sugiyama et al.

Original: (2008; 2007); Shimodaira (2000).
Revised: (2008; 2007); Shimodaira (2000).

Original: This allows us to extend the solutions to minimize the induced risk too.
Revised: This allows us to extend the solutions to minimize the induced risk too.

Original: Nonetheless we will highlight additional computational challenges.
Revised: Nonetheless we will highlight additional computational challenges.

Original: We focus on the covariate shift setting.
Revised: We focus on the covariate shift setting.

Original: The scenario for target shift is similar.
Revised: The scenario for target shift is similar.

Original: For covariate shift, recall that earlier we derived the following fact:
Revised: For covariate shift, recall that earlier we derived the following fact:

Original: [Equation]
Revised: [Equation]

Original: This formula informs us that a promising solution that uses ω x ( h ) to perform reweighted ERM.
Revised: This formula informs us that a promising solution that uses ω x ( h ) to perform reweighted ERM.

Original: Of course, the primary challenge that stands in the way is how do we know ω x ( h ) .
Revised: Of course, the primary challenge that stands in the way is how do we know ω x ( h ) .

Original: There are different methods proposed in the literature to estimate ω x ( h ) when one has access to D ( h ) Zhang et al.
Revised: There are different methods proposed in the literature to estimate ω x ( h ) when one has access to D ( h ) Zhang et al.

Original: (2013b); Long et al.
Revised: (2013b); Long et al.

Original: ; Gong et al.
Revised: ; Gong et al.

Original: How any of the speciﬁc techniques work in our induced domain adaptation setting will be left for a more thorough future study.
Revised: How any of the speciﬁc techniques work in our induced domain adaptation setting will be left for a more thorough future study.

Original: In this section, we focus on explaining the computational challenges even when such knowledge of ω x ( h ) can be obtained for each model h being considered during training.
Revised: In this section, we focus on explaining the computational challenges even when such knowledge of ω x ( h ) can be obtained for each model h being considered during training.

Original: Though ω x ( h ) , (cid:96) ( h ; x, y ) might both be convex with respect to (the output of) the classiﬁer h , their product is not necessarily convex.
Revised: Though ω x ( h ) , (cid:96) ( h ; x, y ) might both be convex with respect to (the output of) the classiﬁer h , their product is not necessarily convex.

Original: Consider the following example:
Revised: Consider the following example:

Original: Example 1 ( ω x ( h ) · (cid:96) ( h ; x, y ) is generally non-convex) .
Revised: Example 1 ( ω x ( h ) · (cid:96) ( h ; x, y ) is generally non-convex) .

Original: Let X =
Revised: Let X =

Original: Let the true label of each x ∈ X
Revised: Let the true label of each x ∈ X

Original: be y ( x ) = 1 (cid:0) x ≥ 1 2 (cid:1) .
Revised: be y ( x ) = 1 (cid:0) x ≥ 12 (cid:1) .

Original: Let (cid:96) ( h ; x, y ) = 1 2 ( h ( x ) − y ) 2 , and let h ( x ) =
Revised: Let (cid:96) ( h ; x, y ) = 12 ( h ( x ) − y ) 2 , and let h ( x ) =

Original: x (simple linear model).
Revised: x (simple linear model).

Original: Notice that (cid:96) is convex in h .
Revised: Notice that (cid:96) is convex in h .

Original: Let D be the uniform distribution, whose density function is f D = (cid:26) 1 , 0 < x ≤0 , otherwise .
Revised: Let D be the uniform distribution, whose density function is f D = (cid:26) 1 , 0 < x ≤0 , otherwise .

Original: Notice that if the training data is drawn from D , then h is the linear classiﬁer that minimizes the expected loss.
Revised: Notice that if the training data is drawn from D , then h is the linear classiﬁer that minimizes the expected loss.

Original: Suppose that, since h rewards large values of x , it induces decision subjects to shift towards higher feature values.
Revised: Suppose that, since h rewards large values of x , it induces decision subjects to shift towards higher feature values.

Original: In particular, let D ( h ) have density function
Revised: In particular, let D ( h ) have density function

Original: [Equation]
Revised: [Equation]

Original: [Equation] x ≤0 , otherwise Then for all x ∈ X , ω x ( h ) =
Revised: Then for all x ∈ X , ω x ( h ) =

Original: f D ( h ) ( x ) f D ( x ) = 2 x. Notice that ω x ( h ) =
Revised: f D ( h ) ( x ) f D ( x ) = 2 x. Notice that ω x ( h ) =

Original: 2 x is convex in h ( x ) = x .
Revised: 2 x is convex in h ( x ) = x .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: which is clearly non-convex.
Revised: which is clearly non-convex.

Original: Nonetheless, we provide sufﬁcient conditions under which ω x ( h ) · (cid:96) ( h ; x, y ) is in fact convex:
Revised: Nonetheless, we provide sufﬁcient conditions under which ω x ( h ) · (cid:96) ( h ; x, y ) is in fact convex:

Original: Proposition D.1.
Revised: Proposition D.1.

Original: Suppose ω x ( h ) and (cid:96) ( h ; x, y ) are both convex in h , and ω x ( h ) and (cid:96) ( h ; x, y ) satisfy ∀ h, h (cid:48) , x, y : ( ω x ( h ) − ω x
Revised: Suppose ω x ( h ) and (cid:96) ( h ; x, y ) are both convex in h , and ω x ( h ) and (cid:96) ( h ; x, y ) satisfy ∀ h, h (cid:48) , x, y : ( ω x ( h ) − ω x

Original: ( h (cid:48) ))
Revised: ( h (cid:48) ))

Original: · ( (cid:96) ( h ; x, y ) − (cid:96) ( h (cid:48) ; x, y )) ≥ 0 .
Revised: · ( (cid:96) ( h ; x, y ) − (cid:96) ( h (cid:48) ; x, y )) ≥ 0 .

Original: Then ω x ( h ) · (cid:96) ( h ; x, y ) is convex.
Revised: Then ω x ( h ) · (cid:96) ( h ; x, y ) is convex.

Original: Proof.
Revised: Proof.

Original: Let us use the shorthand ω ( h ) :
Revised: Let us use the shorthand ω ( h ) :

Original: = ω x ( h ) and (cid:96) ( h ) :=
Revised: = ω x ( h ) and (cid:96) ( h ) :=

Original: (cid:96) ( h ; x, y ) .
Revised: (cid:96) ( h ; x, y ) .

Original: To show that ω ( h ) · (cid:96) ( h ) is convex, it sufﬁces to show that for any α ∈ [0 , 1] and any two hypotheses h, h (cid:48)
Revised: To show that ω ( h ) · (cid:96) ( h ) is convex, it sufﬁces to show that for any α ∈ [0 , 1] and any two hypotheses h, h (cid:48)

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: By the convexity of ω ,
Revised: By the convexity of ω ,

Original: [Equation]
Revised: [Equation]

Original: and by the convexity of (cid:96) ,
Revised: and by the convexity of (cid:96) ,

Original: [Equation]
Revised: [Equation]

Original: Therefore it sufﬁces to show that[ α
Revised: Therefore it sufﬁces to show that

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: h (cid:48)

Original: [Equation]

Original: [Equation]

Original: [ α · (cid:96) ( h )

Original: [Equation]

Original: (1 − α )

Original: [Equation]
Revised: [Equation]

Original: (cid:96) ( h (cid:48)

Original: [Equation]

Original: ] − α ·

Original: [Equation]

Original: ( h ) · (cid:96) ( h ) + (1 − α ) · ω

Original: ( h (cid:48) )

Original: · (cid:96) ( h (cid:48) )

Original: ≤⇔ α ( α − 1) · ω ( h ) (cid:96) ( h ) − α ( α − 1) · [ ω ( h ) (cid:96) ( h (cid:48) )

Original: + ω ( h (cid:48) )

Original: (cid:96) ( h )]

Original: + α ( α − 1) · ω ( h (cid:48) ) (cid:96) ( h (cid:48) )

Original: ≤⇔ α ( α − 1) · [ ω ( h ) − ω

Original: ( h (cid:48) )] · [ (cid:96) ( h ) − (cid:96) ( h

Original: [Table]
Revised: [Table]

Original: )] ≤⇔ [ ω ( h ) − ω ( h (cid:48) )] · [ (cid:96) ( h ) − (cid:96) ( h

Original: [Table]

Original: )] ≥

Original: By the assumed condition, the left-hand side is indeed non-negative, which proves the claim.
Revised: By the assumed condition, the left-hand side is indeed non-negative, which proves the claim.

Original: This condition is intuitive when each x belongs to a rational agent who responds to a classiﬁer h to maximize her chance of being classiﬁed as +1 :
Revised: This condition is intuitive when each x belongs to a rational agent who responds to a classiﬁer h to maximize her chance of being classiﬁed as +1 :

Original: For y = +1 , the higher loss point corresponds to the ones that are close to decision boundary, therefore, more − 1 negative label points might shift to it, resulting to a larger ω x ( h ) .
Revised: For y = +1 , the higher loss point corresponds to the ones that are close to decision boundary, therefore, more − 1 negative label points might shift to it, resulting to a larger ω x ( h ) .

Original: For y = − 1 , the higher loss point corresponds to the ones that are likely mis-classiﬁed as +1, which “attracts” instances to deviate to.
Revised: For y = − 1 , the higher loss point corresponds to the ones that are likely mis-classiﬁed as +1, which “attracts” instances to deviate to.

Original: D. C HALLENGES DUE TO THE LACK OF ACCESS TO DATA
Revised: D. C HALLENGES DUE TO THE LACK OF ACCESS TO DATA

Original: We discuss the challenges in performing induced domain adaptation.
Revised: We discuss the challenges in performing induced domain adaptation.

Original: In the standard domain adaptation settings, one often assumes the access to a sample set of X , which already poses challenges when there is no access to label Y after the adaptation.
Revised: In the standard domain adaptation settings, one often assumes the access to a sample set of X , which already poses challenges when there is no access to label Y after the adaptation.

Original: Nonetheless, the literature has observed a fruitful development of solutions Sugiyama et al.
Revised: Nonetheless, the literature has observed a fruitful development of solutions Sugiyama et al.

Original: ; Zhang et al.
Revised: ; Zhang et al.

Original: (2013b)
Revised: (2013b)

Original: ; Gong et al.
Revised: ; Gong et al.

Original: One might think the above idea can be applied to our IDA setting rather straightforwardly by assuming observing samples from D ( h ) , the induced distribution under each model h during the training.
Revised: One might think the above idea can be applied to our IDA setting rather straightforwardly by assuming observing samples from D ( h ) , the induced distribution under each model h during the training.

Original: However, we often do not know precisely how the distribution would shift under a model h until we deploy it.
Revised: However, we often do not know precisely how the distribution would shift under a model h until we deploy it.

Original: This is particularly true when the distribution shifts are caused by human responding to a model.
Revised: This is particularly true when the distribution shifts are caused by human responding to a model.

Original: Therefore, the ability to “predict” accurately how samples “react” to h plays a very important role Ustun et al.
Revised: Therefore, the ability to “predict” accurately how samples “react” to h plays a very important role Ustun et al.

Original: Indeed, the strategic classiﬁcation literature enables this capability by assuming full rational human agents.
Revised: Indeed, the strategic classiﬁcation literature enables this capability by assuming full rational human agents.

Original: For a more general setting, building robust domain adaptation tools that are resistant to the above “prediction error” is also going to be a crucial criterion. 
Revised: For a more general setting, building robust domain adaptation tools that are resistant to the above “prediction error” is also going to be a crucial criterion. E D ISCUSSIONS O N P ERFORMING D IRECT I NDUCED R ISK M INIMIZATION

Revised: In this section, we provide discussions on how to directly perform induced risk minimization for our induced domain adaptation setting.

Revised: We ﬁrst provide a gradient descent based method for a particular label shift setting where the underlying dynamic is replicator dynamic described in Section 5.3.

Revised: Then we propose a solution for a more general induced domain adaptation setting where we do not make any particular assumptions on the undelying distribution shift model.

Revised: E. G RADIENT DESCENT BASED METHOD

Revised: Here we provide a toy example of performing direct induced risk minimization under the assumption of label shift with underlying dynamics as the replicator dynamics described in Section 5.3.

Revised: Setting Consider a simple setting in which each decision subject is associated with a 1-dimensional continuous feature x ∈ R and a binary true qualiﬁcation y ∈ {− 1 , +1 } .

Revised: We assume label shift setting, and the underlying population dynamic evolves the replicator dynamic setting described in

Revised: Section 5.3.

Original: {+We consider a simple threshold classiﬁer, where ˆ Y = h ( x ) = 
+}
Revised: {+We consider a simple threshold classiﬁer, where ˆ Y = h ( x ) = 
+}

Revised: [ X ≥ θ ] , meaning that the classiﬁer is completely characterized by the threshold parameter θ .

Revised: Below we will use ˆ Y and h ( X ) interchangeably to represent the classiﬁcation outcome.

Revised: Recall that the replicator dynamics is speciﬁed as follows:

Revised: [Equation]

Revised: [Equation]

Revised: where E D S [ Fitness ( Y )]

Revised: =

Revised: Fitness ( Y = y ) P D S ( Y = y )

Revised: + Fitness ( Y = − y )

Revised: (1 − P D S ( Y = y )) .

Revised: Fitness ( Y = y ) is the ﬁtness of strategy Y =

Revised: y , which is further deﬁned in terms of the expected utility U y, ˆ y of each qualiﬁcation-classiﬁcation outcome pair ( y, ˆ y ) :

Revised: [Equation]

Revised: where U y, ˆ y is the utility (or reward) for each qualiﬁcation-classiﬁcation outcome combination.

Revised: P ( X | Y = y ) is sampled according to a Gaussian distribution, and will be unchanged since we consider a label shift setting.

Revised: We initialize the distributions we specify the initial qualiﬁcation rate P D S ( Y = +1) .

Revised: To test different settings, we vary the speciﬁcation of the utility matrix U y, ˆ y and generate different dynamics.

Revised: Formulate the induced risk as a function of h To minimize the induced risk, we ﬁrst formulate the induced risk as a function of the classiﬁer h ’s parameter θ taking into account of the underlying dynamic, and then perform gradient descent to solve for locally optimal classiﬁer h ∗ T .

Revised: Recall from Section 5, under label shift, we can rewrite the induced risk as the following form:

Revised: [Equation]

Revised: where p ( h ) =

Revised: P D ( h ) ( Y = +1) .

Revised: Since E D S [ (cid:96) ( h ; X, Y ) | Y

Revised: = +1] and E D S

Revised: [ (cid:96) ( h ; X, Y )

Revised: | Y = − 1] are already functions of both h and

Revised: D S , it sufﬁces to show that the accuracy on D ( h ) , p ( h ) =

Revised: P D ( h ) ( Y = +1) , can also be expressed as a function of θ and D S .

Original: {+To see this, recall that for a threshold classiﬁer ˆ Y = 
+}
Revised: {+To see this, recall that for a threshold classiﬁer ˆ Y = 
+}

Revised: [ X > θ ] , it means that the prediction accuracy can be written as a function of the threshold θ and target distribution D ( h ) :

Revised: [Equation]

Revised: where P ( X | Y

Revised: =

Revised: y ) remains unchanged over time, and P D ( h ) ( Y = y ) evolves over time according to Equation (22), namely

Revised: [Equation]

Revised: Notice that ˆ Y is only a function of θ , and U y, ˆ y are ﬁxed quantities, the above derivation indicates that we can express P D ( h ) ( Y = y ) as a function of θ and D S .

Revised: Plugging it back to Equation (23), we can see that the accuracy can also be expressed as a function of the classiﬁer’s parameter θ , indicating that the induced risk can be expressed as a function of θ .

Revised: Thus we can use gradient descent using automatic differentiation w.r.t θ to ﬁnd a optimal classiﬁer h ∗ T that minimize the induced risk.

Revised: Experimental Results Figure 7 shows the experimental results for this toy example.

Revised: We can see that for each setting, compared to the baseline classiﬁer h ∗ S , the proposed gradient based optimization procedure returns us a classiﬁer that achieves a better prediction accuracy (thus lower induced risk) compared to the accuracy of the source optimal classiﬁer.

Revised: E. G ENERAL S ETTING : I NDUCED R ISK M INIMIZATION WITH B ANDIT F EEDBACK

Revised: In general, ﬁnding the optimal classiﬁer that achieves the optimal induced risk h ∗ T is a hard problem due to the interactive nature of the problem (see, e.g. the literature of performative prediction Perdomo et al.

Revised: (2020) for more detailed discussions).

Revised: Without making any assumptions on the mapping between h and D ( h ) , one can only potentially rely on the bandit feedbacks from the decision subjects to estimate the inﬂuence of h on D ( h ) : when the induced risk is a convex function of the classiﬁer h ’s parameter θ , one possible approach is to use the standard techniques from bandit optimization (Flaxman et al., 2004) to iteratively ﬁnd induced optimal classiﬁer h ∗ T .

Revised: The basic idea is: at each step t = 1 , · · · , T , the decision maker deploy a classiﬁer h t , then observe data points sampled from D ( h t ) and their losses, and use them to construct an approximate gradient for the induced risk as a function of the model parameter θ t .

Revised: When the induced risk is a convex function in the model parameter θ , the above approach guarantees to converge to h ∗ T , and have sublinear regret in the total number of steps

Revised: T .

Revised: The detailed description of the algorithm for ﬁnding h ∗ T is as follows:

Revised: Algorithm 1: One-point bandit gradient descent for performative prediction

Revised: Result: return θ T after T rounds θ 1 ← 0 foreach time step t ← 1 , . .

Revised: , T do

Revised: Sample a unit vector u t ∼ Unif( S ) θ + t ← θ t + δu t

Revised: Observe data points z 1 , . .

Revised: z n t ∼ D ( θ + t ) (cid:101) IR

Revised: ( θ + t ) ← 1 n t (cid:80) n t i =1 (cid:96) ( z i ; θ + t )

Revised: ˜

Revised: g t ( θ t ) ← dδ (cid:101) IR

Revised: ( θ + t ) · u t (cid:46) ˜

Revised: g t ( θ t ) is an approximation of

Revised: ∇ θ (cid:98) IR ( θ t ) θ t +1 ← Π (1 − δ )Θ ( θ t − η ˜ g t ( θ t ))

Revised: (cid:46) Take gradient step; project onto (1 − δ )Θ := { (1 − δ ) θ | θ ∈ Θ } end

Revised: F R EGULARIZED T RAINING

Revised: In this section, we discuss the possibility that indeed minimizing regularized risk will lead to a tighter upper bound.

Revised: Consider the target shift setting.

Revised: Recall that p ( h ) :

Revised: =

Revised: P D ( h ) ( Y = +1) and we have for any proper loss function (cid:96) :

Revised: [Equation]

Revised: Suppose p < p ( h ∗ T ) , now we claim that minimizing the following regularized/penalized risk leads to a smaller upper bound.

Revised: [Equation]

Revised: where in above D uniform is a distribution with uniform prior for Y .

Revised: We impose the following assumption:

Revised: • The number of predicted +1 for examples with Y = +1 and for examples with Y

Revised: = −are

Revised: monotonic with respect to α .

Revised: Consider the easier setting with (cid:96) = 0-1 loss.

Revised: Then

Revised: [Equation]

Revised: The above regularized risk minimization problem is equivalent to

Revised: [Equation]

Revised: Recall the upper bound in Theorem 5.1:

Revised: [Equation]

Revised: With a properly speciﬁed α > 0 , this leads to a distribution with a smaller gap of | p (˜ h S ) − p ( h ∗ T ) | , where ˜ h S denotes the optimal classiﬁer of the penalized risk minimization - this leads to a smaller

Revised: Term 1 in the bound of Theorem 5.1.

Revised: Furthermore, the induced risk minimization problem will correspond to an α s.t.

Revised: α ∗ =

Revised: p ( h ∗ T ) − p 0 ., and the original h ∗ S corresponds to a distribution of α = 0 .

Revised: Using the monotonicity assumption, we will establish that the second term in Theorem 5.1 will also smaller when we tune a proper α .

Revised: G D ISCUSSION ON THE TIGHTNESS OF OUR THEORETICAL BOUNDS

Original: {+General Bounds in Section 
+}
Revised: {+General Bounds in Section 
+}

Revised: For the general bounds reported in Section 3, it is not trivial to fully quantify the tightness without further quantifying the speciﬁc quantities of the terms, e.g. the H divergence of the source and the induced distribution, and the average error a classiﬁer have to incur for both distribution.

Revised: This part of our results adapted from the classical literature in learning from multiple domains Ben-David et al.

Revised: The tightness of using H -divergence and other terms seem to be partially validated therein.

Revised: Bounds in Section 4 and Section 5 For more speciﬁc bounds provided in Section 4 (for covariate shift) and Section 5 (target shift), however, it is relatively easier to argue about the tightness: the proofs there are more transparent and are easier to back out the conditions where the inequalities are relaxed.

Revised: For example, in Theorem 5.1, the inequalities of our bound are introduced primarily in the following two places: 1) one is using the optimiality of h ∗ S on the source distribution.

Revised: 2) the other is bounding the statistical difference in h ∗ S and h ∗ T ’s predictions on the positive and negative examples.

Revised: Both are saying that if the differences in the two classiﬁers’ predictions are bounded in a range, then the result in Theorem 5.1 is relatively tight.