Stateful Strategic Regression

1 Introduction . Automated decision-making tools increasingly assess individuals to determine whether they qualify for life-altering opportunities in domains such as lending [ 27 ] , higher education [ 32 ] , employment [ 41 ] , and beyond . These assessment tools have been widely criticized for the blatant disparities they produce through their scores [ 43 , 3 ] . This overwhelming body of evidence has led to a remarkably active area of research into understanding the societal implications of algorithmic/data-driven automation . Much of the existing work on the topic has focused on the immediate or short-term societal effects of automated decision-making . ( For example , a thriving line of work in Machine Learning ( ML ) addresses the unfairness that arises when ML predictions inform high-stakes decisions [ 18 , 22 , 31 , 8 , 1 , 16 , 11 ] by defining it as a form of predictive disparity , e.g. , inequality in false-positive rates [ 22 , 3 ] across social groups . ) With the exception of several noteworthy recent articles ( which we discuss shortly ) , prior work has largely ignored the processes through which algorithmic decision-making systems can induce , perpetuate , or amplify undesirable choices and behaviors . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Our work takes a long-term perspective toward modeling the interactions between individual decision subjects and algorithmic assessment tools . We are motivated by two key observations : First , algorithmic assessment tools often provide predictions about the latent qualities of interest ( e.g. , creditworthiness , mastery of course material , or job productivity ) by relying on imperfect but observable proxy attributes that can be directly evaluated about the subject ( e.g. , past financial transactions , course grades , peer evaluation letters ) . Moreover , their design ignores the compounding nature of advantages/disadvantages individual subjects accumulate over time in pursuit of receiving favorable assessments ( e.g. , debt , knowledge , job-related skills ) . To address how individuals respond to decisions made about them through modifying their observable characteristics , a growing line of work has recently initiated the study of the strategic interactions between decision-makers and decision-subjects ( see , e.g. , [ 15 , 26 , 36 , 30 , 21 ] ) . This existing work has focused mainly on the short-term implications of strategic interactions with algorithmic assessment tools—e.g. , by modeling it as a single round of interaction between a principal ( the decision-maker ) and agents ( the decision-subjects ) [ 30 ] . In addition , existing work that studies interactions over time assumes that agents are myopic in responding to the decision-maker ’ s policy [ 4 , 42 , 38 , 15 ] . We expand the line of inquiry to multiple rounds of interactions , accounting for the impact of actions today on the outcomes players can attain tomorrow . Our multi-round model of principal-agent interactions . We take the model proposed by Kleinberg and Raghavan [ 30 ] as our starting point . In Kleinberg and Raghavan ’ s formulation , a principal interacts with an agent once , where the interaction takes the form of a Stackelberg game . The agent receives a score y = f ( ✓ , o ) , in which ✓ is the principal ’ s choice of assessment parameters , and o is the agent ’ s observable characteristics . The score is used to determine the agent ’ s merit with respect to the quality the principal is trying to assess . ( As concrete examples , y could correspond to the grade a student receives for a class , or the FICO credit score of a loan applicant . ) The principal moves first , publicly announcing her assessment rule ✓ used to evaluate the agent . The agent then best responds to this assessment rule by deciding how to invest a fixed amount of effort into producing a set of observable features o that maximize his score y. Kleinberg and Raghavan characterize the assessment rules that can incentivize the agent to invest in specific types of effort ( e.g. , those that lead to real improvements in the quality of interest as opposed to gaming the system ) . We generalize the above setting to T > 1 rounds of interactions between the principal and the agent and allow for the possibility of certain effort types rolling over from one step to the next . Our key finding is that longer time horizon provides the principal additional latitude in the range of effort sequences she can incentivize the agent to produce . To build intuition as to why repeated interactions lead to the expansion of incentivizable efforts , consider the following stylized example : Example 1.1 . Consider the classroom example of Kleinberg and Raghavan where a teacher ( modeled as a principal ) assigns a student ( modeled as an agent ) an overall grade y based on his observable features ; in this case test and homework score . Assume that the teacher chooses an assessment rule and assigns a score y = ✓TETE + ✓HWHW , where TE is the student ’ s test score HW is his homework score , and ✓T , ✓HW 2 R are the weight of each score in the student ’ s overall grade . The student can invest effort into any of three activities : copying answers on the test , studying , and looking up homework answers online . In a one-round setting where the teacher only evaluates the student once , the student may be more inclined to copy answers on the test or look up homework answers online , since these actions immediately improve the score with relatively lower efforts . However , in a multiple-round setting , these two actions do not improve the student ’ s knowledge ( which impacts the student ’ s future grades as well ) , and so these efforts do not carry over to future time steps . When there are multiple rounds of interaction , the student will be incentivized to invest effort into studying , as knowledge accumulation over time takes less effort in the long-run compared to cheating every time . We revisit this example in further detail in Appendix A . Summary of our findings and techniques . We formalize settings in which the agent ’ s effort investment today can accumulate over time in the form of an internal state—impacting both his future rewards and that of the principal . We characterize the Stackelberg equilibrium of the resulting game and provide novel algorithmic techniques for computing it . We begin by establishing that for the principal , the class of all linear assessment policies remains as powerful as the larger class of all monotonic assessment policies . In particular , we prove that if there exists an assessment policy that can incentivize the agent to produce a particular sequence of effort profiles , there also exists a linear assessment policy which can incentivize the exact same effort sequence . We then study the equilibrium computation problem , which in general involves optimizing non-convex objectives . Despite the initial non-convexity , we observe that when the problem is written as a function of the agent ’ s incentivized efforts , the principal ’ s non-convex objective becomes convex . Moreover , under a common assumption on agent ’ s conversion mapping from efforts to observable features , the set of incentivizable effort policies is also convex . Given this structure , we provide a polynomial-time algorithm that directly optimizes the principal ’ s objective over the set of incentivizable effort policies , which subsequently recovers agent ’ s and principal ’ s equilibrium strategies . Even though prior work [ 39 , 40 ] has also taken this approach for solving other classes of non-convex Stackelberg games , our work has to overcome an additional challenge that the agent ’ s set of incentivizable efforts is not known a-priori . We resolve this challenge by providing a membership oracle ( that determines whether a sequence of agent efforts can be incentivized by any assessment policy ) , which allows us to leverage the convex optimization method due to Kalai and Vempala [ 28 ] . Our analysis reveals several intriguing insights about the role of repeated interactions in shaping the long-term outcomes of decision-makers and decision subjects : For example , we observe that with multiple rounds of assessments , both parties can be better off employing dynamic/time-sensitive strategies as opposed to static/myopic ones . Crucially , perhaps our most significant finding is that by considering the effects of multiple time-steps , the principal is significantly more effective at incentivizing the agent to accumulate effort in her desired direction ( as demonstrated in Figure 1 for a stylized teacher-student example ) . In conclusion , our work addresses two critical gaps in the growing literature on the societal impacts of automated decision-making–by focusing on longer time horizons and accounting for the compounding nature of decisions individuals receive over time . 1.1 Related work . A growing line of work at the intersection of computer science and social sciences investigates the impacts of algorithmic decision-making models on people ( see , e.g. , [ 25 , 44 , 34 , 15 ] ) . As we outline below , significant attention has been devoted to settings in which decision-subjects are strategic and respond to the decision-maker ’ s choice of assessment rules . Liu et al . [ 34 ] and Kannan et al . [ 29 ] study how a utility-maximizing decision-maker may respond to the predictions made by a predictive rule ( e.g. , the decision-maker may interpret/utilize the predictions a certain way or decide to update the model entirely . ) Mouzannar et al . [ 37 ] and Heidari et al . [ 23 ] propose several dynamics for how individuals within a population may react to predictive rules by changing their qualifications . Dong et al . [ 15 ] , Hu et al . [ 26 ] , Milli et al . [ 36 ] address strategic classification—a setting in which decision subjects are assumed to respond strategically and potentially untruthfully to the choice of the predictive model , and the goal is to design classifiers that are robust to strategic manipulation . Generalizing strategic classification , Perdomo et al . [ 38 ] propose a risk-minimization framework for performative predictions , which broadly refers to settings in which the act of making a prediction influences the prediction target . Incentive-aware learning [ 45 , 2 ] is another generalization that , at a high-level , seeks to characterize the conditions under which one can train predictive rules that are robust to training data manipulations . Two additional topics that are conceptually related to our work but differ in their motivating problems and goals are adversarial prediction and strategy-proof regression . The adversarial prediction prob- lem [ 6 , 13 ] is motivated by settings ( e.g. , spam detection ) in which an adversary actively manipulates data to increase the false-negative rate of the classifier . Adversarial predictions have been modeled and analyzed as a zero-sum game [ 13 ] or a Stackelberg competition [ 6 ] . Strategyproof/truthful linear regression [ 14 , 12 , 9 ] offers mechanisms for incentivizing agents to report their data truthfully . As mentioned earlier , many of our modeling choices closely follow Kleinberg and Raghavan [ 30 ] . Below , we provide a summary of Kleinberg and Raghavan ’ s results and briefly mention some of the recent contributions following their footsteps . While much of prior work on strategic classification views all feature manipulation as undesirable [ 15 , 26 , 36 ] , Kleinberg and Raghavan made a distinction between feature manipulation via gaming ( investing effort to change observable features in a way that has no positive impact on the quality the principal is trying to measure ) and feature manipulation via improvement ( investing effort in such a way that the underlying characteristics the principal is trying to measure are improved ) . Their model consists of a single round of interaction between a principal and an agent , and their results establish the optimality and limits of linear assessment rules in incentivizing desired effort profiles . Several papers since then have studied similar settings ( see , e.g. , Miller et al . [ 35 ] , Frankel and Kartik [ 19 ] ) with goals that are distinct from ours . ( For example , Frankel and Kartik find a fixed-point assessment rule that improves accuracy by under-utilizing the observable data and flattening the assessment rule . ) Finally , we mention that principle-agent games [ 33 ] are classic economic tools to model interactions in which a self-interested entity ( the agent ) responds to the policy/contract enacted by another ( the principal ) in ways that are contrary to the principle ’ s intentions . The principal must , therefore , choose his/her strategy accounting for the agent ’ s strategic response . Focusing on linear strategies is a common practice in this literature [ 24 , 7 , 17 ] . For simplicity , we present our analysis for linear assessment rules , but later show that the class of all linear assessment policies is equally as powerful as the class of all monotone assessment policies ( Theorem 3.4 ) .

Summary: This paper studies a theoretical model of a multi-round assessment game between a principal and an agent. This model is an extension of Kleinberg and Raghavan (2020) to the multi-time-step setting where the efforts of an agent can accumulate benefits from one time step to the next. It studies the equilibrium computation problem under simplifying linearity assumption. Its main finding is that in the multi-step case, the principal can incentivize a wider range of effort profiles than in the single time step case.