Accounting for AI and Users Shaping One Another: The Role of Mathematical Models
Authors: Sarah Dean
Evan Dong
Meena Jagadeesan
Liu Leqi
Venue: Accepted by TMLR ()

Abstract: As AI systems enter into a growing number of societal domains, these systems increasingly shape and are shaped by user preferences, opinions, and behaviors. However, the design of AI systems only sometimes accounts for how AI and users shape one another. In this survey paper, we discuss the development of formal interaction models which mathematically specify how AI and users shape one another. Formal interaction models can be leveraged to (1) specify interactions for implementation, (2) monitor interactions through empirical analysis, (3) anticipate societal impacts via counterfactual analysis, and (4) control societal impacts via interventions. The design space of formal interaction models is vast, and model design requires careful consideration of factors such as style, granularity, mathematical complexity, and measurability. Using content recommender systems as a case study, we critically examine the nascent literature of formal interaction models with respect to these use-cases and design axes. More broadly, we call for the community to leverage formal interaction models when designing, evaluating, or auditing any AI system which interacts with users.

Published in Transactions on Machine Learning Research (06/2025)
Accounting for AI and Users Shaping One Another: The Role of Mathematical Models
Sarah Dean
sdean diean@cornell.oo
Deportment of Computer Science
Cornell University
Evan Dong
Along@ess.ornellee edu
Department of Computer Science
Cornell University
Meena Jagadeesan
mjjopuddesson@@eekkllee.. edu
Department of EECS
University of Colifornia, Berkeley
Liu Leqi
Aigiliiuuttaas.comm edu
Deportment of IROM
Universily of Tezus, Austin
Reviewed on OpenReview: https: // openrevie.. net/ forum? id= Uhr4
Abstract
As AI systems enter into a growing number of socictal domains, these systems increasingly shape and are shaped by user preferences, opinions, and behaviors. However, the design of Al systems only sometimes accounts for how Al and users shape one another. In this survey paper, we discuss the development of formal interaction models which mathematically specify how AI and users shape one another. Formal interaction models can be leveraged to (1) specify interactions for implementation, (2) monitor interactions through empirical analysis, (3) anticipate societal impacts via counterfactual analysis, and (4) control societal impacts via interventions. The design space of formal interaction models is vast, and model design requires careful consideration of factors such as style, granularity, mathematical complexity, and measurability. Using content recommender systems case study, critically cxamine the nascent literature of formal interaction models with respect to these use-cases and design axes. More broadly, we call for the community to leverage formal interaction models when designing, evaluating, or auditing any Al system which interacts with users!
1 Introduction
Machine learning enables the creation of platforms and products which are data-driven, adaptive, and intelligent. When these Al systems are deployed into the world, it is common for these systems to shape their users, and turn for users to shape these systems. For example, consider content recommender systems, which provide algorithmic curation and personalization. Content recommender systems interact with both viewers and creators; these interactions can shape viewer preferences (Leqi al. (023) and shape the landscape of content available on the platform (Meyerson) (0122). which in turn affects downstream recommendations. Another example is user-facing decision-support systems, which have been integrated into
hiring pipelines (Commission el al. 2022 credit score assessment (Sadok et al. 2022 and resource allocation
Authors in alphabetical order.
2 An system refers any algorithmic learning system which transforms data predictions, decisions, or other output forms (e.g., through optimizing loss function).

---

Published in Transactions on Machine Learning Research (06/2025)
Obermeyer et al. 2019: : these systems have shaped the actions and outcomes of users Björkegren al. 20200. More recently, large language models (L.MMs) have started to interact with ecosystems of consumers and service-providers, which has created new avenues for shaping the world at large (Pan et al. 2024).
The fact that AI systems and users shape one another can lead to unanticipated ramifications for platforms, users, and society at large. In established domains, these consequences have been well documented. For example, content recommender systems can lead to clickbait at the level of the platform (Meyerson 2012). addiction at the level of individual users (Hasan et al., 2018), and growing radicalization at the level of society (Haroon et al., 2022). As another example, user-facing decision-support systems (e.g., in hiring pipelines, credit score assessment, and resource allocation) can load to stratogic behavior (Rosenfeld 2024 and can affect access to opportunities and fairness objectives (Reader et al. 2022 Pagan et al.) 2023). In fact, even in nascent domains, AI systems have already started to exhibit such dynamics. For example, LLM-basod chatbots can produce emotionally charged and aggressive language (Perrigo) 2023), and text generation models enable the mass production of SEO spam (Notopoulos) 2024). Furthermore, the prospect of aulonomous cars brings these concerns from the digital to the physical world, where smart routing algorithms may disrupt driving norms or traffic flow (Sadigh et al. 2016), and incentivize antagonistic behaviors by humans (Paul 2023).
The core conceptual problem is that AT system design and evaluation does not appropriately account for how these systems and users shape one another. In fact, this conceptual problem surfaces in many learning paradigms. For example, in supervised learning settings, the traditional learning paradigm of training on a large static dataset fails to account for how the AI system changes the enviromment in which operates. At deployment, the AI system can trigger a unforescen distribution shift, which can lead to unintended impacts on performance and on society at large. To some extent, approaches based reinforcement learning (RL) attempt to account for such shifts. However, the state representations, reward function, and transition models-winng are implemented in simulators or learned from data in model-based RL often fail to capture koy aspects of interactions and dynamics between the AI system and users. As a rcsult, at deployment, the RL policy can also shape the enviroument in unforeseen ways and similarly trigger unintended impacts (Gilberl)
et al. 2022 2023
Towards addressing the shortcomings of AI system design and evaluation, we highlight emerging strategy based on the development of formal interaction models, i.e., mathematical models which formalize how AI and users shape one another. We define a formal interaction model as a coupled dynamical system between the Al system and users (Section 2.1 Figure lb Formal interaction models can be leveraged to improve AI system design and evaluation, as illustrated by the following use-cases: (1) specifying interactions for implementation, (2) monitoring interactions through empirical analysis, (3) anticipating societal impacts via counterfactual analysis, and (4) controlling societal impacts via interventions (Section 2.2 The design space of formal interaction models is vast, and there is no fint-slzo-fits-all" approach for designing model: which approach is most appropriate is use-case dependent. Model design thus requires careful consideration of design axes such as style, granularity, mathematical complexity, and mcasurability (Section 2.3
Taking content recommendation as a case study, we critically examine the nascent literature of formal interaction models with respect to these use-cases and design axes (Section 3). The formal interaction models in this literature capture how recommender systems can shape users from viewer preferences to content creator strategies but different works formalize these models in varying languages (e.g., nonlinear dynamics, game theory, bandits, and behavioral psychology). To compare the formal interaction models proposed in different works, we cast each formal interaction model within the coupled dynamical systems language (Figure D). We identify that the literature primarily focuses on anticipating and controlling societal impacts, suggesting that there is room for future work focused on specifying and monitoring interactions. We further observe that the models of each interaction type are fairly homogeneous in style, and that models of viewer interactions and creator interactions have been largely separate; this motivates future work which broadens the scope of these models.
SSee Section 2.1 for additional discussion of how standard learning paradigms fail appropriately account for how AI and users shape one another. The discussion is specialized content recommender systems for concreteness.
2

---

Published in Transactions on Machine Learning Research (06/2025)
Dynamical systems model
ya system (f")
xP
Environment
User .0'
User ( P'gg)
(F.g]
(a) Model and environment interaction
(b) Formal interaction model as coupled dynamical system
Figure 1: Left: a dynamical system where model interacts with an unspecified environment. Right: a formal interaction model as coupled dynamical system between an AI system and users. The variable x captures the internal state, the function f captures the state update, the function 9 captures the measurement equation, the variable u denotes inputs, the variable denotes outputs (which also serve as inputs, in the case of the coupled dynamical system), and the variable wo captures noise or other disturbance (Section 2.1
More broadly, we advocate for formal interaction models principled approach to account for AT systems and users shaping one another across wide range of domains (c.g., content recommender systems, LLM-based ecosystems, decision-support systems, aulonomous vehicles, etc). We are inspired by the success of formal interaction models in mechanism design and physical systems (Appendix A), and we expect that formal interaction models-if appropriately leveraged- can similarly contribute to AI system design. We call for the community to leverage formal interaction models when building, evaluating, deploying, and auditing AI systems: only when we properly account for how users and AT systems shape one another will we be able to build reliable Al systems with positive societal ramifications.
2 Formal Interaction Models
We investigate the role of formal (mathematical) models in accounting for interactions between AI systems and users. We focus class of models which we call formal interaction models which specify how an AI system shapes the internal state of individual users and how these users shape the AT system. We provide an overview of formal interaction models (Section 2.1 present use cases of formal interaction models (Section 2.2). and discuss how model design requires careful considcration along several axes (Section 2.3
2.1 Overview of Formal Interaction Models
We first justify the need for formal interaction models, using content recommender systems a motivating example. Then, we define formal interaction models both conceptually and as a coupled dynamical system. Finally, we give examples of formal interaction models for user-AT interactions across many domains.
Motivating Example: Content Recommendation. While content recommendation was initially designed help individuals navigate a "deluge" of articles, songs, and videos sent over email lists (Konstan al., 1997 Hill et al.) 1995: Shardanand & Macs 1995), modern content recommendation is performed by large online platforms. On today's online platforms- such YouTube, Netflix, and TikTok-viess
and creator populations are within closed gardens and can be shaped by the algorithms deployed by the platform (Gillespie 2014 Seaver 2022). In particular, the specifics of a content recommendation model can shape the internal state of viewers such as viewers preferences, behaviors, or participation decisions; the content recommendation model can also shape the internal state of creators such as creator decisions about what content create, which affects the content landscape. Morcover, these changes viewer preferences and the content landscape alter the characteristics of the online platform, contributing to societal level impacts. (See Section 3 for further discussion of how the recommender system shape viewers and creators, and vice versa.)
However, standard approaches for designing content recommender systems fail to appropriately account for how AI and users shape one another. For example, consider lwo-lower embedding models based on matrix factorization (e.g. Hu et al. 2008) or deep learning variants (e.g. Yi et al. 2019 Yang et al.) 2020 where the recommender systems Iearns emboddings for content and embeddings for viewer preferences, and assigns recommendations based on these embeddings. The main issue is that these architectures implicitly treat the viewer preference embeddings and content embeddings fixed, and thus fail to account for how the
3

---

Published in Transactions on Machine Learning Research (06/2025)
recommender system shapes viewer preferences and the creator incentives. As another example, consider model-free reinforcement learning (RI.) approaches. One issue is that RI. approaches make extensive use of simulators (e.g. [e et al., 2019) which often make simplifying assumptions about how recommender systems shape viewers and creators. In fact, most simulators entirely ignore how the recommender system shapes creator decisions, and treat the landscape of content available on the platform as fixed. Even when simulators do account for how the recommender system shapes viewers, RL approaches optimize aggressively for reward signals and can unintentionally manipulate viewer preferences in the process (e.g. Carroll et al., 2022)
As result of these conceptual shorteomings, deployed recommender systems have induced unintended socictal ramifications for the platform (e.g., performance degradation due to clickbait (Meyerson 2012 users (e.g,, addiction (Hasan et al., 2018)), and even society at large (e.g., radicalization qHaroon et al., 2022 ). This highlights the need for better accounting of how the recommender system and users shape one another (Boutilier et al.) 2023 Leqi & Dean 2022). As we describe in more detail below, formal interaction models are designed capture these dynamics.
High-Level Definition of Formal Interaction Model. Formal interaction models are mathematical models which capture how an AI system shapes users and how users shape the Al system. An AT system refers to any algorithmic or learning system which transforms data into predictions, decisions or other output forms (c.g., through optimizing a loss function). A user refers to any human who interacts with the Al system (directly or indirectly). Formal interaction models can draw principles from many disciplines (e-g.; dynamical systems, online learning, game theory, industrial organization, behavioral psychology, causal modelling, etc.), but nonetheless share a common high-level structure that we describe below.
Formal interaction models specify causal relationships between impuls, oulpuls, and internal slates of the AT system and users. In particular, a formal interaction model specifies "state updates": how the outputs (or decisions) of the Al system affect the internal state of each individual user, and how the outputs of each individual user (e.g., observed behaviors) affect the state of the AI system. A formal interaction model also specifies the relationship between the user's internal state and output (and similarly for the Al system). The key component of formal interaction model is how the Al system outputs shape the internal state of users, and we thus focus on models in which the internal state of each user nontrivially affected by the outpuls of the Al system.
We also focus on formal interaction models which capture users at the individual level, taking complementary perspective from population- approaches to mat hematical modelling of Al-user interactions. Examples of population-level approaches are the framework of performative prediction (Perdomoo et al.) 2020 (which formalizes how a prodictor can shape the distribution of users in supervised lcarning) and the framework of reinforcement learning (which abstracts population dynamics into an "environment" in which sequential decision-making occurs (Sutton & Barto (018)). Tndividual-level models specify granular behaviors, and they can be aggregated and analyzed at the population level to draw conclusions about societal impacts (see e.g.
how individual-leve strategic behavior results in population-level distribution shift (Perdomo et al. 2020 ).
Returning to the content recommender system example, the AT system refers to the recommender system, and "users" can refer to either viewers or creators. The internal state of viewers captures viewer preferences or other behaviors, and the internal state of creators captures internal decisions about what content to create or whether to participate on the recommendation platform. The outputs of the recommender system are recommendations, the outputs of viewers are observed behaviors such clicks, and the outputs of creators are uploaded content or participation oulcomes. We describe formal interaction models for content recommendation in more detail in Section 3
Definition of a Formal Interaction Models as a Coupled Dynamical System. To further distill the definition of a formal interaction model, we cast a formal interaction model as a coupled dynamical system. This formulation serves as useful unifying language describe many different mathematics ical models for
4

---

Published in Transactions on Machine Learning Research (06/2025)
Creator of (f",g")
Recommender System (f'gg)
Viewer x² gf"
y
Creator 2 2/2 (f":g")
Figure 2: A formal interaction model for content recommendation as a coupled dynamical system between the recommender system (x",y", f".'"") viewers (x","", f","") and creators (v","" f"f", The variable x captures the internal state, the function f captures the state update, the function g captures the measurement equation, and the variable y denotes outputs and inputs (Section 2.1 and Section 3
user-AT interactions. We first provide a brief overview of the dynamical systems language and then cast. formal interaction models within this language
Following Willems (1989); a (causal) dynamical system is a mathematical object describing the evolution of various quantities over discrete time steps 2 (Figure La]. Its definition is grounded by observable quantities, which come in two types: inputs, denoted by u, come from possibly unexplained environment; and outpuls, denoted by y, come from the dynamical system model. It is often convenient to introduce an internal state variable z which is internal and evolves over time (2))120- At its most general, dynamical systems model specifies relationships between sequences of inputs (u))20 and sequences of outputs (ye ( The only universal restriction is that the model is causal, meaning that the output yt is not affected by the future inputs uy for greater than 1.
A dynamical systems model can be formalized by an internal state variable æ (how the state is represented), state update f (how the state is affected by inputs) and measurement equation 9 (how the state affects outputs). The functions/ I and recursively define the possible outputs of a system in terms of its inputs and an additional time-dependent disturbance variable wA which accounts for non-stationary, stochastic, or adversarial effects. More specifically, the internal state is updated according to I4+1 f(I,, Me wx) and the outputs are updated according to y/ = g(I,, un, W)).
Using the dynamical systems language, formal interaction models can be expressod as coupled dynamical system-in -i.e, multiple, interconnected dynamical systems- as shown in Figure 1b Each user i's behavior is captured by dynamical systems model (z',f",g'), and the Al system's behavior is also captured by a dynamical systems model ("", These dynamical systems are coupled in that the outputs yf of the Al's dynamical system are the inpuls ut of each user's dynamical system, and the oulpuls yi of each user's dynamical system together form the impul uh+1 of the AT system's 's dynamical system. The key component of a formal interaction model is the state update f² for each user, which captures how the Al system shapes the user. To capture meaningful dynamics, this state update must nontrivially change with the output y².
For the special case of content recommendation, a formal interaction model is a coupled dynamical system between the recommender system, viewers, and creators (Figure 2). We defer extensive discussion of content recommendation to Section 3 but provide simplified example here to build intuition.
Example 1 Consider a model of user opinion in a video recommendation setting. For simplicity, consider videos representing opinions along a single aris, e.g. plant-based us. meat-based diets. Let y ER, denote the user watch time and ya R denote the recommended video, parametrized by this single opinion aris.
Let the user's internal state x' E IR be their own scalar opinion. Suppose that watchtime depends on the exlent to which the user agrees with the video. Further suppose that the strength of user opinions tends to decay, but that they "assimilate" the opinions contained in the recommended videos. This con be formalized by the state update 1 = f ) 0.9 2 + 0.1 yf and the measurement equation
"The dynamical systems language (Willens 1989) is appealing for describing formal interaction models for the following it provides clear formulation for cause and effect, is well-suited applications due to its focus on mcasurable quantitics, and yet it retains sufficient generality to describe many different types of models. Note that if opinion were independent of videos (e.g. if 2i_1 as formal interaction model (or alternatively, would be "trivial" formal interaction model).
all time steps), then this model would not counted
5

---

Published in Transactions on Machine Learning Research (06/2025)
vi = Ri w/) ovv xr. activation function mapping to [0. 1].
w/) where wi is noise accounting for erogenous factors and a is some
The recommender system's internal state is an estimate of user preferences based on observed watchtimes. Recommendations y." are chosen to mazimize predicted watchtime. As a result, users watch videos that reinforce their eristing opinions (so long as the recommender adequately learns from data). At a population level, this induces polarization, wherein user opinions are positive or negative but nonzero, e.g. in favor of plant meat based diets rather than omnivorous.
Examples of Formal Interaction Models for User-AI Interactions. Due to the widespread ability of Al systems to shape users, formal interaction models are starting to be developed across many domains h For example, in content recommendation, a nascent body of work (which we survey in Section 3 proposes models for how content recommender systems can shape viewers and creators. As another example, formal interaction models in the strategic clossification literature (e.g. Hardt al. 2016 Kleinberg & Raghavan 2019 Rosenfeld 2024) capture how a decision rule incentivizes individual users to change their features to try to reccive positive prediction. Moreover, other works have proposed formal interaction models for how
fairness interventions shape dynamics (e.g. Pagan et al.] 2023 Reader et al.] 2022 Liu et al. 2018). how
autonomous vehicles shape driver behavior (e.g. |Sadigh et al. 2016), and how foundation model shapes the
finetuning or prompting decisions of service providers (c.g. Jagadecsan et al. 2023
Laufer et al. 2023
While these nascent literatures are promising, formal interaction models for user-Al systems are still limited in scope and have not been sufficiently leveragod in the design and evaluation of AI systems. This highlights the continued need to develop (and expand the scope of) formal interaction models for user-AI systems.
2.2 Use Cases of Formal Interaction Models
We argue that formal interaction models can play an instrumental role in specifying interactions for implemen- tation, monitoring interactions through empirical analysis, anticipating societal impacts via counterfactual analysis, and controlling societal impacts via interventions. Our argument draws inspiration from Abebe et al. 2020) which outlines the roles of computer science for social change.
Specify interactions for implementation. Developing formal interaction models forces the rescarch community to fully specify Al-user interactions, so that these models can be implemented to empirically capture feedback effects.
Implementing formal interaction models can improve the empirical evaluation and design of AI systems across many different learning paradigıs. For example, in the context of model-based RL, a formal interaction model-in in particular, each user's internal state x² and state update f'-can be incorporated into the RL state representation and the transition probabilities between states. As another example, the quantities z¹ and f of a formal interaction model can also be implemented in envéronment simulators to empirically capture how users are shaped by the actions taken by the AI systems. These enviromment simulators can be used to train and evaluate model-free RL (where the transitions probabilities no longer need to be specified ahead of time) and also to evaluate supervised learning-basee systems (which do not explicitly account for interactions with users). As an example, in fair machine learning, D'Amour al. (2020) design simulators to evaluate how feedback dynamics affect. fairness in the context of bank loans, college admissions, and allocation of attention. In content recommendation, (2020) Mladenov et al. (2021); Gao el al. (2023) Dellayel et al. (2023) designs simulators to evaluate how recommender system performance is affected by interactions with users over time
emerging body of work by Rohde et al. 20018) le et al. (2019); Krauth et al.
Monitor interactions through empirical analysis. Developing formal interaction models can make it. more tractable to monitor interaction patterns between AI systems and users in real-world deployments.
Even well before the emergence of modern Al systems, formal interaction models were developed in classical decision-makinn systems, for example in mechanism and market design as well as in physical systems (Appendix A Thhese simulation frameworks could serve as useful testbed implement and test the formal models that we survey in our case study (Section
6

---

Published in Transactions on Machine Learning Research (06/2025)
Formal interaction models can be leveraged to monitor the strength and character of interactions between AI systems and users in real-world environments. These mathematical models can surface relevant constructs to monitor and can also formalize these constructs as variables (e.g., finite-dimensional parameters) which can be estimated from data about real-world interactions. The estimated parameters can be assessed to determinc whether the interactions are strong (or weak) in a given enviromment. and to shed light. on other qualitative properties (such as which users are most affected) [ As an example, in strategic classífication, Björkegren et al. (020)) measure manipulation costs in large field experiment in Kenya, which provides insight into the strength and character of strategic interactions. As another example, in content recommender systems, Milli 2021) ) measure how well different engagement. signals (and combinations of signals) capture value, and
ct al.110
Aridor et al. (2023) measure the causal effect of recommendations on viewer beliefs about content quality.
Even observing that a formal interaction model does not capture real-world interactions itself provides valuable information. In particular, falsifying a model demonstrates that either the structure of interactions might be different than anticipated (which motivates adjusting the model) or that the model might miss important interaction patterns (which motivales augmenting or expanding the model). As an example, in strategic classification, JJgadeesan et al. (2021) observed that the standard agent models fail to capture distribution shifts observed in practice and also have limited prescriptive value. As another example, Leqi et al. 2023 tested the fixed reward distribution assumption in culti--rmed-bandii bandit-based recommender systems and found it to be violated. In such cases, falsifying existing models through empirical analysis can encourage the community to refine models to better capture empirical realities.
Anticipate societal impacts via counterfactual analysis. Developing formal interaction models enables the community to anticipate societal impacts driven by Al-user interactions in counterfactual environments (e.g., where AT algorithm changes or user interactions change).
Formal interaction models can be leveraged to characterize when societal impacts occur in existing domains. For example, analyses of formal interaction models (via theoretical analyses or simulations) can reveal how the algorithm parameters and the broader environment together influence the strength and nature of the societal impact. These analyses can enable the community to anticipate the impacts of Al-user interactions in counterfactual environments which could arise with changes to the algorithm or users. As an example of an empirical analysis, in fair machine learning, D' et al (020) uses the simulations to compare and assess the long-term fairness of different algorithms; as an example of a thcoretical analysis, in strategic classification, Kleinberg & Raghavan (2019) characterize how the scoring rule shapes how much effort agents invest in improvement versus gaming.
Formal interaction models can help forecast what socictal impacts will occur in emerging domains. This is particularly important when new technologies are introduced into the world. We expect that some principles underlying formal interaction models in existing domains can be translated to new domains, which could help anticipate potential societal impacts of new technologies before they occur. For example, to forecast behaviors about emerging LLM-based ecosystems, Jagadecsan et al. (2023) : Laufer et al. (2023) develop formal interaction models which borrow inspiration from industrial organization and bargaining games.
Control societal impacts via interventions. Developing formal interaction models can help control societal impacts through either through improving algorithm design or highlighting avenues for broader sociotechnical interventions (such as public policy).
Formal interaction models can surface new metrics that can be optimized by supervised learning or RT.-basdd algorithms to control societal impacts. Certain socictal impacts may be directly mcasurable by a single parameter of the mathematical model, whereas other societal impacts may arise as complex combination of multiple parameters. We expect that crafting new metrics which capture these societal impacts could benefit the design of objectives, which is classically an ad-hoc process that requires trial-and-crror. As an example, for the Twitter platform, Milli et al. (2021) propose metrics to measure value rather than engagement. As broader research agenda, [Boutilier et al. (2023) propose leveraging tools from mechanism design to recommender systems design which optimize downstream user welfare and ecosystem health.
8 Parameter estimation common use-case of formal interaction models in econometrics and empirical industrial organization.
7

---

Published in Transactions on Machine Learning Research (06/2025)
Formal interaction models can also help highlight avenues for broader sociotechnical interventions. It may be intractable to control certain societal impacts purely via algorithm design, or the flatform may not be incentivized to take responsibility. In such cases, formal interaction models may inform how to disrupt Al-user interactions with non-algorithmic design choices (c.g.. UI choices) or with public policy. For example, seminal work in fair machine learning (e.g. Kleinberg et al. 2017 Chouldechova 2017) used mathematical models (though not formal interaction models) to show inherent tradcoffs between different fairness definitions for any scoring rule; these works illuminates the boundaries of what is achievable with technical interventions (Abebe et al., 2020). We expect that the formal interaction models similarly be leveraged to inform the limits of technical interventions and ultimately inform the design of broader sociotechnical interventions.
Measurability
Style Dynamical systems model
Use Case
Style Gramularity Complexity Measurability
Environment
Specifying
(J.9)
Monitoring
Anticipating
Granularity
Complexity
Controlling
(a) Axes of model design.
(b) Model design guidelines.
Figure 3: Left: A breakdown of which design axes correspond to which components of the formal model. Righl: Guidelines for model design along different axes based on intended use case. Checkmarks indicate which axes are especially important for each use case.
2.3 Axis of Varying Models: Guidelines for Design Choices
Formal interaction models vary along several design axes axes-style style, granularity, mathematical complexity, and measurability creates vast design space of possible models. There is no "ono-size-fts--ll" approach: which approach is most suitable depends on the particular use case. In this section, we grapple with how model design requires careful consideration along style, gramularit, mathematical complexity, and measurability. We outline each design axis, and offer guidance on the design decisions tailored to each use-case. Figure summarizes the design axes and the guidance on design decisions.
Axis I: Style. The style of a formal interaction model depends the disciplines)) which model design is based. For example, nonlinear dynamics-based models often have roots in control theory, reward models are often based on psychology and behavioral economics, and game-thhery-based models often incorporate perspectives from microeconomics and industrial organization. The model style particularly affects how the user internal state x' is represented and how the state update fr is specified. For example, while these quantities are explicitly defined in nonlinear dynamics-bason models, they are only implicit in the strategy and utility function defined by game theery-based models. Example
(linear) dynamics-based model.
Axis II: Granularity. The granularity of a model describes the level of detail to which model is specified, ie. the specificity of the relationships between inputs, outputs, and states. Examples of highly granular models are fully parametric models (e.g. linear models). At the other extreme, low-granularity models might only specify only inputs and outputs and not incorporate any further structure. At the middle of the spectrum are models with structure beyond their input and output, but do not fully specify funcciionl forms. This includes graphical models (such as those used in casual modelling) which illustrate dependencies, but either do not specify any functional relationships or place additive structure which allows for flexibility in each component. Example] I is a granular model, since the relationships are exactly specified with given parameters.
Axis III Mathematical Complexity. The complexity of a model captures the overall mathematical intricacy of the model. Key factors which affect model complexity are the intricacy of the state update f (for how AI outputs shape the internal state of users) and the intricacy of the measurement equation g (which determines user outputs). Complex models often have comprehensive set of variables and flexible
8

---

Published in Transactions on Machine Learning Research (06/2025)
functional form with the goal of capturing all factors of interest. At the other extreme, simpler models are usually more interpretable and focus only on first-order factors. Example 1 is a low complexity model with scalar variables and one dimensional functions.
Axis IV: Measurability. The measurability of a model captures how feasible it is to measure the mathematical constructs of the formal interaction model with real-world observable data. Examples of highly measurable models include parametric models based observable constructs such watch time and clicks. On the other hand, some models incorporate constructs (e.g., the psychological state of a user) that may be naturally hard impossible to observe. Nonetheless, the model still may be partially measurable if the model is identifiable and observable data can be used to infer the underlying parameters. However, the model may not be measurable if the functional form of a model is unidentifiable, in which case observable data insufficient to infer model parameters. When a formal interaction model contains unobservable and unidentifiable constructs, this can serve as a call to action for measuring additional variables or re-conceptualizing design of the system (Jacobs & Wallach 2021). Example | is measurable due to the simple specified relationship between observable watchtime and unobscrved opinion.
Guidelines for Design Choices. Given that the four axes described above style, granularity, mathematical complexity, and measurability create a vast design space for formal interaction models, we reflect on best practices for designing a formal interaction model. One challenge is that the most appropriate design choices highly depend on the intended use-case of the model, and there thus no one-size-fits all approach for model design. We thus offer high-level guidelines tailored to each use-case, highlighting which axes we believe are most important to consider for each use-case.
Specifying inteructions: When specifying interactions for the purpose of designing simulators, we view gramularity as the most important axis, and complexity relevant. In particular, to empirically capture feedback effects, it is necessary to design a granular model with sufficient complexily to fully specify how the users react to the AI system outputs.
Monitoring interactions: When monitoring Al-user interactions, designing a measurable model is key. In particular, the constructs that are monitored need to be measurable from observable data, both to determine the strength or character of Al-user interactions and to validate (or falsify) the model. Furthermore, model style should be tailored to the aspects of interactions that one aims to monitor. For example, to monitor strategic behaviors of users, a game theory-based model would be helpful; to monitor the preference shifts of user, non-linear dynamics based models may be more appropriate.
Anticipate societal impacts: When anticipating societal impacts, designing mathematically simple model can help capture the salient aspects of the ecosystem and uncover the drivers of a societal impact. Relatedly, model style should be tailored to which factors that drive the societal impact of interest. For example, nonlincar jyyamicc-based models are particularly appropriate for capturing convergence and evolution over time. As another example, reward models in ouline decision--akinn settings (e.g,, multi-armed bandits) are well-suited to capturing how decisions impact perceived rewards. As yet another example, game-theory models are helpful for capturing the equilibrium state when many users compete with each other.
Control societal impacts: When controlling societal impacts via algorithm design, the choices along mulliiple azes impact the practicality and robustness of an algorithmic intervention. For example, measurabilit billity affects whether the construct of interests can be optimized from the observable data available to the AI system. Furthermore, mathematica ical simplicity affects the tractability of obtaining provable guarantees. Finally, the granularity affects the robustness of interventions, since models which specify exact functional forms can be subject to model misspecifications.
Taking these guidelines into account, Example i is perhaps best suited to anticipating societal impacts related to polarization over time, as is a simple dynamics- based model. In contrast, its complexity likely too low to be meaningfully used in specifying, monitoring, or controlling, since opinion shifts may depend on interactions between many factors, rather than single scalar value.
9

---

Published in Transactions on Machine Learning Research (06/2025)
Motivating Phenomenon
Style
Use Case
Viewer Preference
echo chambers, polarization, boredom diversity, clickhait over-conssmption, retention retention miscellaneous
dynamics, bandit game theory; bandit
anticipating, controlling anticipating, controlling anticipating, controlling antiripating, controlling anticipating. controlling
Creator Content
Viewer Participation
hehavioral econ, bandit game theory, handit
Creator Partiripation
Other
game theory, behavioral econ
Table 1: Summary of formal interaction models in content rocommendation, addressing viewer preferences, creator content, participation, and other behaviors.
3 Case Study: Content Recommendation
In this section, we return to our molivating example of content recommendation (Section 2.1 study. Our goal is to critically examine emerging line of work which grapples with the ways that content recommender systems shape viewers and creators. To compare these models and place them on a common ground, we cast, these models within the unifying framework of coupled dynamical systems (Section 2.1 Figure 2 We investigate how the formal interaction models proposed in this line of work capture the use-cases outlined in Section 2.2 We also consider model design along the axes outlined in Section 2.3 Table summarizes our analysis.
a case
We carry out this analysis for formal interaction models where viewer or creator internal states are shaped by the recommender system. This includes viewer preferences (Section 3.1), creator content (Section 3.2) and other behaviors (Section 3.3 For each model, we focus on how the AI system shapes users: in the dynamical systems language, this is the state x' and state update I for each user. Note that in this analysis, we reflect on lines of work as a whole, rather than surveying specific models in individual papers. The collection of papers included in the analysis was generated as follows: first, a small set of key papers was identified; then, it was expanded following the citation graph; finally, only papers defining a non-trivial formal interaction model where included. The collection includes papers published through carly 2024.
Our analysis uncovers several limilations of this nascent literature and reveals interesting opportunities for future work (Section 3.4). First, in terms of use-cases, we find that most of the focus has been on anticipating societal impacts and controlling societal impacts; this highlights the need for future work focused on specifying interactions or monitoring interactions. Second, in terms of model design, we find that the models are fairly homogeneous in style within each interaction type, and that models of viewer interactions and creator interactions have been largely separate; this highlights an opportunity broaden the scope of these models.
Recall that we do not consider dynamics models where the user state does not change (e.g. Chaney et al. 2018 Mansoury et al. 2020) to be formal interaction models (Section 2.1 Such models do still induce feedback loops because viewers provide the recommender system with feedback about their utility from recommended content, and the recommender system uses viewer feedback to estimate viewer utilities. However, even though the recommender system's embeddings of viewer preferences can change, the viewers' internal states (i.e., preferences) are not changing. These models are thus outside of the scope of our case study.
3.1 Viewer Preferences
One line of work investigates how recommendations affect viewer preferences and opinions. In these papers, the primary setting is an individual viewer consuming content and responding with feedback such as watch time, clicks, or likes. These works focus on how different recommendation algorithms shape a viewer's internal state, how such state changes result in broader socictal impacts, and how platform can design algorithms to control the negative societal impacts. Below, we present overview and critical examination of these works, and we defer detailed mathematical definitions to Appendix B
Societal impacts. One motivation for this line of work is the unintended consequences of personalized rocommendation. For example, socictal impacts-sun such as echo chambers, polarization, biased assimilation, manipulation (c.g. Lu et al.) [2014] Dean Morgenstern 2022)- can emerge from engagement- based recom- mendations shaping viewer preferences. Moreover, when viewers get bored of content, this can lead to broader
10

---

Published in Transactions on Machine Learning Research (06/2025)
forms of misalignment between the goal of recommender systems and the intended goal of recommender systems and what they actually optimize.
Style of the models. Two dominant styles of models are latent factor- based models (e.g. Dean & Morgenstern 2022) and multi-armed bandits models (e.g. Kleinberg & Immorlica 2018). The latent factor models in the literature capture viewer preference shifts through viewer embedding dynamics, typically taking the perspective that recommendations shape how viewers see the world. The multi-armed bandit models capture preference updates in terms of changes in the reward distributions, often taking the perspective that viewers" preference shifts occur naturally as viewers get bored of content.
Overview of models. While these models differ in the type of preference shifts they examine, there are commonalities among the state definitions, state updates, and measurement functions. The state x² represents viewer preferences or opinions that can be represented as vectors or scalars that are discrete (e.g. Jiang et al.
2019
Agarwal Brown 2023) or continuous (e.g. Dean & Morgenstern, 2022 Curmei et al. 2022 observable
(e.g.
Levine et al. 2017 Kleinberg & Immorlica, 2018 or latent (e.g. Leqi et al. 2021 Porat et al. 2022
The measurable behaviors y" of the viewers are the ratings or clicks. The state update f models viewer
preference changes due to recommendations, as result of exposure to a type of content (e.g. Kalimeris
el al. 2021 Evans & Kasirzadeh 2021). viewers' selections (e.g. Brown & Agarwal
2022 Agarwal & Brown
2023 or their measured behavior (e.g. Dean & Morgenstern 2022). Viewer preferences can shift towards the
recommended content (e.g. Carroll et al. 2022 Krueger al. 2020 or away from rocommendations (c.g. Lu
et al. 2014 The state update can also capture the viewer's memory accumulation of (the quality of) past.
recommended content (e.g. Curmei et al.. 2022), which can result in viewer boredom (e.g. Levine et al. 2017
Pike-Burke & Grunewalder 2019) loyalty (e.g. Krueger et al. 2020). Finally, f" may also have a periodic
structure, capturing viewers' scasonal preference shifts (c.g. Laforgue et al. 2022 Foussoul et al. 2023
Use-cases of these models. In terms of use-cases, this line of work has focused on anticipating and controlling societal impacts. The nonlinear-dynamics models are utilized to identify failure modes of existing recommender systems (e.g., biased assimilation, echo chamber, manipulation) and propose adjustments existing algorithms to address these issues (c.g. Carroll et al. 2022: Evans & Kasirzadch, 2021). Mcanwhile,
the bandit-style models are used to develop recommendation algorithmss atluned to impacts on viewers'
transient preference (e.g. Foussoul et al. 2023 Khosravi et al. 2023
3.2 Creator Content
Shifting focus from viewers to creators, a different line of work investigates how content recommender system shapes (and is shaped by) the landscape of content available on the platform. The high-level mechanism is that content creators compete to appear in recommendations: this means that the recommendation tion functions shapes creator incentives around what content to create, which affects the composition of the content landscape. Below, we present an overview and critical examination of these works, and we defer detailed mathematical definitions to Appendix C
Societal impacts. One motivation for this line of work is societal impacts emerging from recommender system shaping the content landscape. For example, shifts in the content landscape affect viewer satisfaction and platform metrics and contribute to broader phenomena such as polarization and addiction. Thus far, this
line of work has focused on the quality and diversity of content and resulting impact on recommendations: e.g., creator effort in quality (e.g. Ghosh & McA fee 2011), content specialization (e.g. JJgaddesan al.
2022 Eilat & Rosenfeld 2023), creators relying on gaming tricks such as clickbait (e.g. Buening et al. 2023
Immorlica ct al. 2024 and implications for engagement and welfare (c.g. Yao et al. 2023b
Style of the models. These models take the perspective of game theory, formalizing how creators- compete to win recommendations -strategically design their content lo appear in as many recommendations as possible. As is common in game theoretic models, the focus is on the equilibria (which corresponds to fized points of the underlying dynamical system). handful of works have introduced dynamics over time by studying convergence of best-response and better-response dynamics (e.g. Ben- Perat al.) 2020 Hron et al. 2022) and by incorporating the flatform learning over multiple time steps (e.g. Ghosh & Hummel 2013).
11

---

Published in Transactions on Machine Learning Research (06/2025)
Overview of models. While these models take the perspective of game-theory, cast these models within the dynamical systems framework to enable better comparison with the interaction patterns for viewer preferences in Section 3.1 The state zj of each creator j captures the creator's internal decision about what content to create and the output of captures the content that the creator uploads. Across different models, the state space ranges from a finile set. (e.g. Ben-Porat & Tennenholty 2018), to Rgo (e.g. Ghosh & McAfoe 2011), to TR. (e.g. Hron et al., 2022 Jagadecsan et al. 2022), to more abstract sets (e.g. Yao et al. 2023b). The state update f captures the creator's best-response to their utility function. The creator utility function captures the creator's reward from recomendations--wieenne
ranges from exposure (e.g. Ben-Porat & Tennenholtz 2018) to engagement (e.g. Yao et al. 2023a) to abstract functions (e.g. Yao et al. 2023b) and also sometimes captures costs of production (e.g. Ghosh & McAffe 2011). Here, the recommendation function g" implicilly shapes the creator's reward by allecting pecommendations, and different papers study different. recommendation functions.
Use-cases of these models. In terms of use-cases, this line of work has focused on anticipating and controlling socictal impacts. For anticipating socictal impacts, some works theorctically analyze how the engagement and welfare of standard supervised learning and bandit algorithms can degrade under creator competition (e.g. Ghosh & Hummel 2013 Yao et al. 2023a; Hu et al. 2023); other works characterize when socictal impacts (e.g., creator specialization) occur as a function of the recommendation function and marketplace specifics (e.g. Jagadoesan et al., 2022). For controlling socictal impacts, some works consider the endogeneity of the content landscape and design recommender system objectives and learning algorithms which optimize for downstream engagement and welfare (e.g. Buening et al. 2023).
3.3 Other Behaviors
More broadly, recommender systems can also shape other aspects of users beyond viewer preferences and creator content. We briefly summarize a few of these aspects below.
Viewer Participation. One line of work studies how viewer participation is influenced by recommender system algorithms.
Behavior modelled. These models capture how viewers may depart the system (e.g. Ben-Porat et al. 2022 or may regret their time on the platform (c.g. Kleinberg et al.] (022).
Overview of models. The state x" captures whether the viewer stays on the platform or has departed (Huttenlocher et al. 2024 Ben- et al. 2022); or it may represent one of the two consumption processes of a viewer (Kleinberg et al.] 2022). The state update f" determines can be specified by a fixed leaving probability (Ben-Porat et al.) 2022); ; or can be determined by the quality of recommendation content. the previous time step (Huttenlocher et al., 2024 Kleinberg al., 2022). Recommender systems are modeled as bandit matching algorithms.
Use-cases. Formal models developed in this line of work are used to anticipate and control societal impacts: analyzing how recommender systems may affect viewers to stay longer than they had intended and designing algorithms that account for viewer departure behaviors.
Creator participation. Another line of work examines how the recommender system can shape whether creators stay on or leave the platform.
Behavior modelled. These models formalize how creators need to achieve sufficient recommendation exposure or engagement with their content to stay on the platform (e.g. MMadenov al. 2020).
Overview of models. The state zf incorporates the creator's (probabilistic) internal decision about whether to stay on the platform and the the creator's cumulative level of exposure (e.g. Mladenov et al. 2020 Ben-Porat & Torkan 2023). (In some models, the creator state also includes creator decisions about content quality (Goohh & McAfee 2011 Ghosh & Hummel 2013).) The state update fr captures that the creator will leave the platform if they do not. receive sufficient exposure or engagement, over certain
12

---

Published in Transactions on Machine Learning Research (06/2025)
period of time. Across the different spaces, the recommender systems range from full-information (e.g. Mladenov et al. 2020) to bandit algorithm (e.g. Ghosh & Hummel 2013).
Use-cases. This line of work has primarily focused on anticipating and controlling societal impacts: in particular, determining when a recommendation algorithm incentivizes creators leave the platform, and designing recommendations algorithms which retain creators.
Other decisions made by viewers Another line of work examines how the recommender system can shape broader forms of viewer decisions beyond participation.
Behavior modelled. These models formalize viewers deciding to not follow recommendations (e.g. Mansour et al. 2020), strategically deciding what content to engage with to curate their feed (Haupt et al. 2023
Cen et al. ), and switching between different behavioral modes (e.g. Kleinberg et al. 2022 The style of model ranges from game-theorotic to behavioral economics-inspirel.
Overview of models. The state x" in these models captures viewer internal decisions, ranging from a vector-valued decision about which content to engage with (Haupt et al.
2023
Cen et al. 2023 to a
binary decision about whether to follow recommendations MMansour et al.
2020
to a behavioral model
controlled by System 1 or System 2 (Kleinberg et al. 2022). The state update function f" ranges from a best-response to shape the trajectory of future recommendations (Haupt ct al.,
2023 Cen ct al., 2023
implicit probabilistic updates depending on the specifics of the content consumed (Kleinberg et 2022
Use-cases.
This line of work has also primarily focused on anticipating and controlling socictal impacts.
3.4 Discussion and Opportunities for Future Work
analysis revealed that most of the formal interaction models for content recommendation have focused on anticipating and controlling societal impacts. Thus far, this line of work has devoted limited attention to specifying interactions (e.g., for desigming comprehensive and realistic simulators) or monitoring interactions (c.g., for asscssing the characteristics of woaldworld interactions). Part of this results from the disconnect between the community that develops formal models and the community that empirically builds simulators or develops recommender systems. To broaden the use cases of formal models, we believe that a centralized platform that utilizes the common language of coupled dynamical systems-would be helpful for researchers to consolidate formal models and for practitioners to integrate these models into the design and evaluation of recommender systems.
Use cases. Returning to the cases outlined in Section 2.2
Enriching the models for each type of interaction. Our analysis revealed that within each interaction type, the models are fairly homogeneous in style. For example, the viewer preference literature has primarily studied nonlinear dynamics-basee models, and the creator content literature has primarily studied game-theory based models. This homogeneity in model style has implicitly limited the focus of these models: the viewer preferences models has primarily focused the dynamics between the recommender system and individual viewer, whereas the creator content models has primarily focused on how competition between multiple creators affects behavior at equilibrium. An important direction for future work is to enrich the style of models for viewer interactions and creator interactions. This includes investigating how interactions between multiple viewers could affect preferences, for example, by incorporating network structure (c.g. Crandall et al, 2008). This also includes incorporating how (nonlinear) dynamics of creators over time affect the content. landscape as well as incorporating how recommendation biases and prices affect market concentration (e.g.
Calvano et al.
2023
Castellini al. 2023
Interplay between multiple interaction types. Our analysis also revealed that the models of viewer interactions and creator interactions are largely separate. However, in reality, we expect that the interplay between viewer interactions and creator interactions drives broader societal impacts such as polarization or addiction. An important direction for future work is thus to integrate formal interaction models of viewers and formal interaction models of creators into a single coupled dynamical system. Recent work (Huttenlocher et al. 2024) takes step in this direction and simultaneously considers both viewer and creator participation
13

---

Published in Transactions on Machine Learning Research (06/2025)
decisions. We encourage future work to incorporate richer forms of interplay between creator interactions and viewer interactions, which we expect will involve analyzing a complex coupled dynamical system (Figure 2
4 Conclusion and Discussion
In this survey paper, argued for the development of formal interaction models for how Al systems and users shape one another. We defined formal interaction model as any coupled dynamical system between an Al system and individual users which captures how the Al system shapes the internal state of users. We outlined how formal interaction models can be leveraged to (1) specify interactions for implementation, (2) monitor interactions through empirical analysis, (3) anticipate socictal impacts via countorfactual analysis, and (4) control societal impacts via interventions. We also discussed how model design needs to be tailored to the intended use-case, highlighting the role of model style, granularity, mathematical complexity, and measurability. Finally, using recommender systems literature of formal interaction models (e.g., of viewer preference shifts, content creator incentives, and other user behaviors). Using the dynamical systems language to place these models onto a common ground, we identified limitations in the uso-cases and scope of these models, and uncovered opportunities for future work.
study, critically examined the nascent
We expect that formal interaction models appropriately leveraged can significantly improve Al system design and evaluation in any domain where Al systems and users interact with each other. This includes content recommender systems, which implicitly shape viewer preferences, the content landscape, and many other user behaviors (Section 3). This also includes user-facing decision-suupoor systems (integrated into hiring pipelines (Commission et al. 2022), credit score assessment (Sadok et al., 2022), , and resource allocation (Obermeyer et al., 2019)) which can lead to strategic behavior (Rosenfeld 2024) and affect access to opportunitics and fairness objectives (Reader et al. 2022 Pagan et al. 2023). As yet another example, this includes autonomous vehicles, which interact with human drivers and can alfect society-level traffic patterns and safety of pedestrians (Sadigh et al. (0166). In the coming years, we expect that this will also include language models, which are being augmented with external tools and integrated into broader ecosystems, creating new forms of Al-user interactions (Pan al. 2024).
As Al systems rapidly enter into more societal domains, the need for formal interaction models is increasing, but the research community has not prioritized developing and leveraging these models. We call for the research community to realize the promise of formal interaction models and integrate these models into the design and evaluation of AI systems.
5 Acknowledgments
We thank Micah Carroll for many insights into preference dynamics models. We also thank Mihaela Curmei, Alex Pan, and Abhishek Shetty for valuable feedback on draft of this paper. This work was supported in part by NSF CCF 2312774, NSF OAC-2311521, Open Philanthropy AT fellowship, LinkedIn Research Award, and gift from Wayfair.
14

---

Published in Transactions on Machine Learning Research (06/2025)
References
Rediet Abebe, Solon Barocas, Jon M. Kleinberg, Karen Levy, Manish Raghavan, and David G. Robinson. Roles for computing in social change. In FAT* '20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, January 27-30, 2020, pp. 252 220. ACM, 2020.
Arpit Agarwal and William Brown. Online recommendations for agents with discounted adaptive preferences. CoRR, abs/2302.060111 2023.
Guy Aridor, Duarte Gonçalves, Daniel Kluver, Ruoyan Kong, and Joseph A. Konstan. The economics of recommender systems: Evidence from a field experiment on moviclens. In Proceedings of the 24th ACM Conference on Economics and Compulation, EC 2023, London, United Kingdom, July 9-12, 2023, pp. 117.
ACM, 2023.
Omer Ben-Porat and Moshe Tennenholtz. A game-theoretic approach to recommendation systems with strategic content providers. In Advances in Neural Information Processing Systems (NeurIPS), pp. 1118 1128, 2018.
Omer Ben-Porat and Rotem Torkan. Learning with exposure constraints in recommendation systems. In Ying Ding, Jie Tang, Juan F. Sequeda, Lora Aroyo, Carlos Castillo, and Geert-Jan Houben (eds.), Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 May 2023, pp. 3456 3466, 2023.
Omer Ben-Porat, Itay Rosenberg, and Moshe Tennenholtz. Content provider dynamics and coordination in recommendation ecosystems. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neurol Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NearIPS 2020, December 6-12, 2020, virtual, 2020.
Omer Ben-Porat, Lee Cohen, Liu Leqi, Zachary Lipton, and Yishay Mansour. Modeling attrition in recommender systems with departing bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 6079, 2022.
Daniel Björkegren, Joshua F. Blumenstock, and Samsun Knight. Manipulation-proof machine learning. CoRR, abs/ hass/2004.03665, 2020.
Craig Boutilier, Martin Mladenov, and Guy Tennenholtz. Modeling recommender ecosystems: Rescarch challenges at the intersection of mechanism design, reinforcement learning and generative models. CoRR, abs/2300.063355, 2023.
William Brown and Arpil Agarwal. Diversified recommendations for agents with adaptive preferences. Advances in Neural Information Processing Systems, 35:26066 12777766-66777, 2022.
Thomas Kleine Buening, Aadirupa Saha, Christos Dimitrakakis, and Haifeng Xu. Bandits meet mechanism design to combat clickbait in online recommendation. CoRR, abs/2311.15647,, 2023.
Emilio Calvano, Giacomo Calzolari, Vincenzo Denicolò, and Sergio Pastorello. Artificial intelligence, algorith- mic recommendations and competition. Social Science Research Network, May 2023.
Micah D. Carroll, Anca D. Dragan, Stuart Russell, and Dylan Hadfield-Menell. Estimating and penalizing induced preference shifts in recommender systems. In International Conference on Machine Learning, ICML 2022, 17-29 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 2686 2708. PMLR, 2022.
Jacopo Castellini, Amelia Fletcher, Peter L. Ormosi, and Rahul Savani. Recommender systems and competition on subscription-based platforms. Social Science Research Network, April 2023.
Leonardo Cella and Nicolò Cesa-Bianchi. Stochastic bandits with delay-dependent payoffs. In International Conference on Artificial Intelligence and Statistics, pp. 11777 PMT.R 2020.
15

---

Published in Transactions on Machine Learning Research (06/2025)
Sarah H. Cen, Andrew Iyas, and Aleksander Madry, User strategization and trustworthy algorithms. CoRR, 0ts//231..1100 17666, 2023.
Allison J. B. Chaney, Brandon M. Stewart, and Barbara E. Engelhardt. How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 2018.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2):153 163, 2017.
US Equal Employment Opportunity Commission al. The americans with disabilities act and the use of software, algorithms, and artificial intelligence to assess job applicants and employecs, 2022.
David J. Crandall, Dan Cosley, Daniel P. Huttenlocher, Jon M. Kleinberg, and Siddharth Suri. Foedback effects between similarity and social influence in online communitics. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Dala Mining, Las Vegus, Nevada, USA, August 24-27, 2008, pp. 160- 168. ACM, 2008.
Mihaela Curmei, Andreas A Haupt, Benjamin Recht, and Dylan Badfield-Menell Towards psychologically- grounded dynamic preference models. In Proceedings of the 16th ACM Conference on Recommender Systems, pp. 35 48, 2022.
Alexander D' Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, David Sculley, and Yoni Halpern. Fairness is not static: deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference Fairness, Accountability, and Transparency, pp. 525-533 2020.
Sarah Dean and Jamie Morgenstern. Preference dynamics under personalized recommendations. In EC '22: The 23rd ACM Conference on Economics and Compulation, Boulder, CO, USA, July 11 15, 2022, pp- 795 816, 2022.
Romain Deffayet, Thibaut Thonet, Dongyoon Hwang, Vassilissa Lehoux, Jean-Michel Renders, and Maarten de Rijke. SARDINE: A simulator for automated recommendation in dynamic and interactive environments. CoRR, abs/2311.16588 2023.
Itay Eilat and Nir Rosenfeld. Performative recommendation: Diversifying content via strategic incentives. volume 202 of Proceedings of Muchine Learning Research, pp. 9082 9103. PMLR, 2023.
Charles Evans and Atoosa Kasirzadeh. User tampering in reinforcement learning recommender systems. CoRR, hbs/2109.04083, 2021.
Ayoub Foussoul, Vincet Goyal, Orestis Papadigenoponlos, and Assaf Zeevi. Last switch dependent bandits with monotone payoll functions. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 10265 10284. PMLR, 2023.
Chongming Gao, Shiqi Wang, Shijun Li, Jiawei Chen, Xiangnan He, Wenqiang Lei, Biao Li, Yuan Zhang, and Peng Jiang. Cirs: Bursting filter bubbles by counterfactual interactive recommender system. ACM Transactions on Information Systems (TOIS), 42(1), 2023. ISSN 1046-8188.
Arpita Ghosh and Patrick Hummel. Learning and incentives user-generated content: multi-armed bandits with endogenous arms. In Robert D. Kleinberg (ed.), Innovations in Theoreticol Computer Science, ITCS '1, Berkeley, CA, USA, January 9-12, 2013, pp. 233-246. ACM, 2013.
Arpita Ghosh and R. Preston McAfee. Incentivizing high-quality user-generated content. Tn Sadagopan Srinivasan, Krithi Ramamritham, Arun Kumar, M. P. Ravindra, Elisa Bertino, and Ravi Kumar (eds.), Proceedings of the 20th International Conference on World Wide Web, WWW 2011, Hyderabad, India, March 28 April 1, 2011, pp. 137-146 ACM, 2011.
Thomas Krendl Gilbert, Sarah Dean, Tom Zick, and Nathan Lambert. Choices, risks, and reward reports: Charting public policy for reinforcement learning systems. arxio preprint ar 2tl:2222000716 2022.
16

---

Published in Transactions on Machine Learning Research (06/2025)
Thomas Krendl Gilbert, Nathan Lambert, Sarah Dean, Tom Zick, Aaron Snoswell, and Soham Mehta. Reward reports for reinforcement learning. In Proceedings of the 2023 AAAT/ACM Conference on AT, Ethics, and Society, pp. 84 130, 2023.
Tarleton Gillespie. The relevance of algorithms. Media technologies: Essays on communication, materiality, and sociely, (60 2014.
Moritz Hardt, Nimrod Megiddo, Christos H. Papadimitriou, and Mary Wootters. Strategic classification. In Proceedings of the 2016 ACM Conference on Innovalions in Theorelical Compuler Science, Combridge, MA, USA, January 14-16, 2016, pp. 111-122 ACM, 2016.
Muhammad Haroon, Anshuman Chhabra, Xin Liu, Prasant Mohapatra, Zubair Shafig, and Magdalena Wojcieszak. Youtube, the great radicalizer? auditing and mitigating ideological biases in youtube recommendations. CoRR, 0os/2203.11060 2022.
Md Rajibul Hasan, Ashish Kumar Jha, and Yi Liu. Excessive use of online video streamnnng services: Impact of recommender system use, psychological factors, and motives. Computers in Humon Behavior, 80:220-221 228, 2018.
Andreas A. Haupt, Dylan Haddfeld-Menede and Chara Podimata. Recommending to strategic users. CoRR, abs/2332.06655) 2023.
Will Hill, Larry Stead, Mark Rosenstein, and George Furnas. Recommending and evaluating choices in virtual community of use. In Proceedings of the SIGCHI conference on Human factors in compuling systems, pp. 194-201 1995.
Jiri Hron, Karl Krauth, Michael Jordan, Niki Kilbertus, and Sarah Dean. Modeling content creator incentives on algorithm-curated platforms. CoRR, abs/2206.131122 2022.
Xinyan Hu, Meena Jagadecsan, Michael I. Jordan, and Jacob Steinhardt. Incentivizing high-quality content. in online recommender systems. CoRR, abs//3306077477 2023.
Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In Proceedings of the 8th IEEE International Conference on Dala Mining (ICDM 2008), December 15-19, 2008, Pisa, Italy, pp. 263 272. IEEE Computer Society, 2008.
Daniel P. Huttenlocher, Hannah Li, Liang Lyu, Asuman E. Ozdaglar, and James Siderius. Matching of users and creators in two-sided markets with departures. CoRR, abs/2401.00313, 2024.
Eugene Ic, Chih-Wei Hsu, Martin Mladenov, Vihan Jain, Sanmit Narvekar, Jing Wang, Rui Wu, and Craig Boutilier. Recsim: A configurable simulation platform for recommender systems. CoRR, abs/1909.04847, 2019.
Nicole Immorlica, Meena Jagadeesan, and Brendan Lucier. Clickbait vs. quality: How engagement-based optimization shapes the content landscape in online platforms. CoRR, abs/2401.0950, 2024.
Abigail Z Jacobs and Hanna Wallach. Measurement and fairness. In Proceedings of the 2021 ACM conference fairness, accountability, and transparency, pp. 375 385, 2021.
Meena Jagadcosan, Celestine Mendler-Dinneer, and Moritz Hardt. Alternative microfoundations for strategic classification. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, TCML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 4687 4697. PMLR, 2021.
Mcena Jagadeesan, Nikhil Garg, and Jacob Steinhardt. Supply-side equilibria in recommender systems. CoRR, abs//200..18 13489, 2022.
Mcena Jagadeesan, Michael I. Jordan, Jacob Steinhardt, and Nika Haghtalab. Improved bayes risk can yield reduced social welfare under competition. CoRR, 146700 2023.
17

---

Published in Transactions on Machine Learning Research (06/2025)
Ray Jiang, Silvia Chiappa, Tor Lattimore, András György, and Pushmeet Kohli. Degenerate feedback loops in recommender systems. In Vincent Conitzer, Gillian K. Hadfield, and Shannon Vallor (eds.), Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2019, Honolulu, HI, USA, January 27-28, 2019, pp. 383-390. ACM, 2019.
Dimitris Kalimeris, Smriti Bhagat, Shankar Kalyanaraman, and Udi Weinsberg. Preference amplification in recommender systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 805-815, 2021.
Khashayar Khosravi, Renato Paes Leme, Chara Podimata, and Apostolis Tsorvantzis. Bandits with deter- ministically evolving states. CoRR, abs/2307. 11655, 2023.
Jon M. Kleinberg and Manish Raghavan. How do classifiers induce agents to invest effort strategically? In Anna Karlin, Nicole Immorlica, and Ramesh Johari (eds.), Proceedings of the 2019 ACM Conference on Economics and Computation, EC 2019, Phoeniz, AZ, USA, June 24-28, 2019, pp. 825 844. ACM, 2019.
Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. In Christos H. Papadimitriou (ed.), 8th Innovations in Theoretical Computer Science Conference, ITCS 2017, January 9-11, 2017, Berkeley, CA, USA, volume 67, pp. 43:1-43:23, 2017.
Jon M. Kleinberg, Sendhil Mullainathan, and Manish Reghavan. The challenge of understanding what users want: Inconsistent preferences and engagement optimization. In EC 222: The 23rd ACM Conference on Economics and Computation, Boulder, CO, USA, July 11 15, 2022, pp. 29. ACM, 2022.
Robert Kleinberg and Nicole Immorlica. Recharging bandits. In 2018 IEEE 59th Annmal Symposium on Foundations of Computer Science (FOCS), pp. 809-319 IEEE, 2018
Joseph A Konstan, Bradley N Miller, David Maltz, Jonathan L Herlocker, Lee R Gordon, and John Riedl. Grouplens: Applying collaborative filtering to usenet news. Commuurications of the ACM, 40(3):77 87, 1997.
Karl Krauth, Sarah Dean, Alex Zhao, Wenshuo Guo, Mihaela Curmei, Benjamin Recht, and Michael I. Jordan. Do offline metrics predict online performance in recommender systems? CoRR, abs/2011.07931, 2020.
David Krueger, Tegan Maharaj, and Jan Leike. Hidden incentives for auto-induced distributional shift. CoRR, hbss/2009009155 2020.
Pierre Laforgue, Giulia Clerici, Nicolò Cesa-Bianchi, and Ran Gilad-Bachrach. A last switch dependent analysis of satiation and seasonality in bandits. In International Conference Artificial Intelligence and Statistics, pp. 971-990, 2022.
Benjamin Laufer, Jon M. Kleinberg, and Hoda Heidari. Fine-tuning games: Bargaining and adaptation for general-purper models. CoRR, abs/2308 043999 2023.
Liu Leqi and Sarah Dean. Engineering safer recommender system. In Workshop on Responsible Decision Making in Dynamic Environments, 2022.
Liu Leqi, Fatma Kilinc Karzan, Zachary Lipton, and Alan Montgomery. Rebounding bandits for modeling satiation effects. Advances in Neural Information Processing Systems, 34:4003-4011 2021.
Liu Loqi, Giulio Zhou, Fatma Kilzan,Karzan, Zachary Lipton, and Alan Montgomery. A field test of bandit algorithms for recommendations: Understanding the validity of assumptions on human preferences in multi-armed bandits. In Proceedings of the 2023 CHT Conference on Human Factors in Computing Systems, pp. 16, 2023.
Nir Levine, Koby Crammer, and Shie Mannor. Rotting bandits. Advances in neural information processing systems, 30, 2017.
18

---

Published in Transactions on Machine Learning Research (06/2025)
Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In Proceedings of the 35th International Conference on Machine Leurning, TCML, 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 3156-3164 3164. PMLR, 2018.
Wei Lu, Stratis loannidis, Smriti Bhagat, and Laks VS Lakshmanan. Optimal recommendations under attraction, aversion, and social influence. In Proceedings of the 20th ACM SIGKDD international conference Knowledge discovery and data mining, pp. 811-820, 2014.
Yishay Mansour, Aleksandrs Slivkins, and Vasilis Syrgkanis. Bayesian incentive-compatible bandit exploration. Operations Resesearch, 68(4):1132-1161, 2020.
Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. Feedback loop and bias amplification in recommender systems. In CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020, pp- 2145 2148. ACM, 2020.
Eric Meyerson. Youtube now: Why we focus watch time, 2012. URL https: http:/bllo..ouuuuue news-and-eve aanddeeentsyyuuuuueeeaaaeeeeeeee why--wwwywwwweeee focus-focusoor on wwwwatch-timeee
Smitha Milli, Luca Belli, and Moritz Hardt. From optimizing engagement to measuring value. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 714 722, 2021.
Martin Mladenov, Elliot Creager, Omer Ben-Porat, Kevin Swersky, Richard S. Zemel, and Craig Boutilier. Optimizing long-term social welfare in recommender systems: A constrained matching approach. In Proceedings of the 37th International Conference on Machine Learning, 1CML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 6987-6998. PMLR, 2020.
Martin Mladenov, Chih-War Hsu, Vihan Jain, Eugene Ie, Christopher Colby, Nicolas Mayoraz, Hubert Pham, Dustin Tran, Ivan Vendrov, and Craig Boutilier. Recsim NG: toward principled uncertainly modeling for recommender ecosystems. CoRR, abs//2188 abs/21103088057 2021.
Katie Notopoulos. Ai spam is already starting to ruin the internet. Business Insider, 2024. URL https: // //ww.bbsinessinsiddrr cow/ htt.a//..............o.ooo.oooan
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(64641:44474466 2019.
Nicolò Pagan, Joachim Baumann, Ezzal Elokda, Giulia De Pasquale, Saverio Bolognani, and Anikó Hannák. A classification of feedback loops and their relation to biases in automated decision-making systems, In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO 2023, Boston, MA, USA, 30 October 2023 1 November 2023, pp. 7:1-7:14. ACM, 2023.
Alexander Pan, Erik Jones, Meena Jagadeesan, and Jacob Steinhardt. Feedback loops with language models drive in-context reward hacking. CoRR, abs/2402.06667,, 2024.
Kari Paul. The rebel group stopping self-driving cars in their tracks one cone at a time The Guardion, 2023.
Juan C. Perdomo, Tijana Zrnic, Celestine Mendler-Dünner and Moritz Hardt. Performative prediction. In Proceedings of the 37th International Conference on Machine Learning, 1CML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 7599-7609. PMLR, 2020.
Billy Perrigo. The new ai-powered bing is threatening users. that's no laughing matter. Time Magazine, 2023.
URL https://time. http:///teeeeccom.eee.
Ciara Piko-Burke and Steffen Grunewalder. Recovering bandits. Advances in Neural Information Processing Systems, 32, 2019.
19

---

Published in Transactions on Machine Learning Research (06/2025)
Lydia Reader, Pegah Nokhiz, Cathleen Power, Neal Patwari, Suresh Venkatasubramanian,, and Sorelle Friedler. Models for understanding and quantifying feedback in societal systems. In Proceedings of the 2022 ACM Conference on Faimess,, Accountabilityy and Transpareney, pp. 1765 1775, 2022.
David Rohde, Stephen Bonner, Travis Dunlop, Flavian Vasile, and Alexandros Karatzoglou. Recogym: A reinforcement learning environment for the problem of product recommendation in online advertising. CoRR, abs/1808.0077 2018.
Nir Rosenfeld. Strategic ML: how to learn with data that 'behaves' In Proceedings of the 17th ACM International Conference on Web Search and Dala Mining, WSDM 2024, Merida, Merico, March 4-8, 2024, 1128 1131. ACM, 2024.
Alvin E. Roth. The Theory and Practice of Market Design. http:///ww..aaaapppeeee////bb/pp 06/ roth- frrch-lecture..o .d 2018.
Dorsa Sadigh, Shankar Sastry, Sanjit A Seshia, and Anca D Dragan. Planning for autonomous cars that leverage effects on human actions. In Robotics: Science and systems, volume 2, pp. 1-9. Ann Arbor, MI, USA, 2016.
Hicham Sadok, Fadi Sakka, and Mohammed El Hadi El Maknouzi. Artificial intelligence and bank credit analysis: A review. Cogent Economics & Finance, 10(1):203262,, 2022.
Eden Saig and Nir Rosenfeld. Learning to suggest breaks: sustainable optimization of long-term user engagement. In International Conference on Machine Learning, pp. 29671-29696. PMLR, 2023.
Nick Seaver. Computing lasle: algorilloms and the makers of music recommendation. University of Chicago Press, 2022.
Upendra Shardanand and Pattic Maes. Social information filtering: Algorithms for automating "word of mouth". In Proceedings of the SIGCHI conference on Human factors in computing syslems, pp. 210 217, 1995.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018
Jan C Willems. Models for dynamics. Dynamics reported: a series in dynamicul systems and their applications,
pp. 171-269, 1989.
Ji Yang, Xinyang Yi, Derck Zhiyuan Cheng, Lichan Hong, Yang Li, Simon Xiaoming Wang, Taibai Xu, and Ed H. Chi. Mixed negative sampling for learning two-tower neural networks in recommendations. In Companion of The 2020 Web Conference 2020, Taipei, Taiwan, April 20 20,24, 2020, pp. 441-447. ACM / IW3C2, 2020.
Fan Yao, Chuanhao Li, Denis Nekipelov, Hongning Wang, and Haifeng Xu. How bad is top-k recommendation under competing content creators? In International Conference Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp- 39674-39701. PMLR, 2023a.
Fan Yao, Chuanhao Li, Karthik Abinav Sankararaman, Yiming Liao, Yan Zhu, Qifan Wang, Hongning Wang, and Haifeng Xu. Rethinking incentives in recommender systems: Are monotone rewards always beneficial? CoRR, hts//2306.07899 2023b.
Xinyang Yi, Ji Yang, Lichan Hong, Derek Zhiyuan Cheng, Lukasz Heldt, Aditee Kumthekar, Zhe Zhao, Li Wei, and Ed H. Chi. Proceedings of the 13th ACM Conference on Recommender Systems, RecSys 2019, Copenhagen, Denmark, Seplember 16-20, 2019, pp. 269 277. ACM, 2019.
neural modeling for large corpus item recommendations. In
20

---

Published in Transactions on Machine Learning Research (06/2025)
A Formal Interaction Models in Classical Decision-Making Systems
Formal interaction models have historically informed the design of classical decision-making systems, well before the emergence of modern AI systems. We summarize two "success stories":
In mechanism and market design, clear understanding of how mechanism impacts participant in- centives via formal interaction model between the mechanism and participants has been crucial to designing efficient marketplaces. For example, in the medical residency match, formal interaction models (capturing how the matching mechanism impacts the incentives of residents and hospitals) informed the design of NRMP match and led to a Nobel Prize in Economics (Roth, 2018). Formal interaction models have also informed market design for kidney exchange and school choice (Roth 2018
In physical systems, clear understanding of how the system affects the physical world formal interaction model, more commonly called "dynamics" model has been crucial for successful decision- making and control. For example, Docision-making in aerospace systems involves navigating and controlling aircrafts and often leverages models of aerodynamics, orbital mechanics, among many others. Control systems frequently leverage these models to calculate optimal paths, manage fuel efficiency, and ensure the stability of the vehicle under various physical conditions. For example, dynamics models have recently been leveraged by SpaceX to develop reusable orbital launch systems
Given the success of formal interaction models in classical decision-making systems, we expect that formal interaction models can similarly drive positive societal impact for Al systems as well.
B Additional details for Preference Shifts in Section 3.1
We discuss each paper, describe how the states and measurement variables are defined, and provide the formal models of state updates and measurement equations. In these models, gd-1 denotes the sphere in dimensions.
We begin by discussing papers that are motivated by phenomena like echo chambers and polarization.
Lu et al. (2014) define for each viewer i the preference state Ei E Rd, the recommendod content vi, E IRd, and the measured rating vh, E R for some latent dimension d. The dynamics capture three possible behaviors: stationary, attraction, and aversation:
~ 14.0
w.p. 01
L1,
E W2- Ui,r
w.p. 02
Ebyle (2% vi.))
Dt=l Wi Wi-rHh
w.p. 0g
Curmei et al. (0022) define for each viewer the preference state x Rd, the recommended content. Vi E IR, and the measured rating vh, E R for some latent dimension d. The define dynamics for an effect "mere exposure"
2i, (1 a)=) av, E = (xt,,.
They propose an additional model of "operant conditioning" with state variable augmented to include a memory variable xi, (pi,, d x IR
6(m,,1 (PE)) Vi,e) mi,41] - [ -1- Dr,t + on 81.
81, = arctan ( 8 - m. (po,)) (wo.)Ter,e) wi.) ElyA, (x))e, v..))
Pi
Dean Morgenstern 2022) define for each viewer the preference state xN, E Stt11, the recommended content Vi, E 1 and the measured rating v", E R for some latent dimension d. The dynamics capture "biased assimilation,, with
J J X D + ne Vie)) Vi Ely (x V.,.)
9 httppppp/aaa...pppp....../////aa
launch launch_aysten, syysten_dvvell
21

---

Published in Transactions on Machine Learning Research (06/2025)
Kalimeris al. (2021) define for each viewer i the recommendation including both content and predicted
score Vi, ($t,t, Gi E R x C and the measured click Vi E {0, 1). The viewer state Fie [0, is memoryless and determined by the predicted score:
(sLL) + (1 (((())(((())))))) - (1
$i,
E E x
$1, S 0
Jiang et al. [2019) define for each viewer the preference state Fi, E IR" the recommended length & slate of content V E I and the measured binary feedback VL, {0, 1] for p discrete piece of content. They study a class of dynamics models wherein the preference for an item increases when it is clicked, and decreases when it is recommended but not clicked:
2 + Lil
va,bi =
A j E vl,
Ely" is increasing in xb
0 +i+tibb xb v+L]= =0
Agarwal Brown (2023 define for each viewer the preference state xy, A(p), the recommended length k slate of content Vi, E ip],, and the clicked item vf. E yi for p discrete piece of content. They study a class of dynamics models:
Ei-o (r + twy V. =cw x (x))))
Brown Agarwal 2022 study a similar model for the case that y =
Krueger et al. 2020 define a global "loyalty" state dy E RRm for each viewer the preference state "y R2, the recommended content vi, E [p], and the consumed content y.,, [p]. A single viewer is active at each timestep with i, softmax(x",,)) Then for this viewer = i,, the loyalty and preference update, and the viewer selects content independent of the top recommendation.
x/e + tnc,,
1 = 1g + a1 21,411
|| + a2ey, l12" Vh,t softmax(=",,)
Carroll et al. 2022) define for each viewer the preference state xi, E Rd, the recommended content Vi, E A(C) for a fixed content set C C= fei,...,cc. Cp} C Rd, and viewer selection behavior Vh, E C. The state update is influenced by a viewer belief over the future available content.
Ecec(H, (c,,,
" w.p. exp(B2(Ac +1-+ (1-A)c p)),
Vi. =cwpp. & [ Tc) Exec (V(,,c))
Evans & Kasirzadeh (2021) define for each viewer i the belief state xy, E [0, 1]s, recommended content type y E 2,, and viewer response Vh, E {0, 1}. For either j.j" = 1,3 (when x",[1]] is largest element) or j.j = 3, (when x;,, [3] is largest clement), 1f,4110 18,, = j'' mninp where pt is sampled of r.v. with E[p.] > v = w.p. [TT,)
Next, survey papers that adopt multi-armed bandits framework where the learner is the recommender system; the arms to pull indicates the recommendation (category) to present to the viewer; and the received reward is the feedback given by the viewer or the viewer utility. We use the notation that for any vector a, ask] indicates the k-th entry of a. In cases where a is a one-hot vector representing the action taken by the learner, its non-zero entry represents the arm being pulled.
In Levine et al. (007)) there are K recommendation categorics/ /arms; and yi,r is a one-hot vector of K dimensions representing the recommendations given viewer at time t. If [[[[k]] 1, arm k is pulled. The viewer state xi, e N has its k-th entry be the number of time times arm &o has been pulled so far. More specifically, "[,+1[A]] [A] = ""[[M) + [[V[
1} and oN [k] O. The expected measurement is the expected reward of pulling an arm: If yf, [k] 1, E[yA,] my (=(,"(())) where my an arm-dependent monotonically decreasing function of the number of arm pulls.
22

---

Published in Transactions on Machine Learning Research (06/2025)
In Kleinberg & Immorlica (2018) the recommendation Vi,, is defined the same as that of Levine al. (0017). The viewer state xh, E No has its k-th entry be the number of time steps elapsed since pulled and x
0. arm: If vf. [k] 1, Elv.' m. xx [+]) The expected measurement is the expected reward of pulling
last time. More specifically, xf, - + 1 if y [k] + 1, 21, [k] 1 if vf,
where my is a concave function of the number of arm pulls. In Pike-Burke & Grunewaldes Ider (2019), me is drawn from a Gaussian process. In Cella & function.
2020), m. is a monotonically increasing
In Leqi et al. 2021 the recommendation vi is defined the same as that of Levine al. 2017): 1. The viewer state Th, N has its k-th entry be the satiation that the viewer has towards arm k. More specifically, + [k] = 7h (*%,ee [f))( and o; [k] 0 where 7 E (0, 1) is the satiation retention factor. The expected measurement is the expected reward of pulling arm: If [(, 1, Ely by Xxx where by E IR is the base reward of arm k and d 2 0 is the exposure influence factor for arm k.
In
et al. 2022 the recommendation sh. is defined the same as that of Levine et al. 2017). The internal state Ti.0 1s a viewer type belonging to [B] (B E N4) sampled from a prior distribution Q. For t 2 1, F,e 1 if v = (i.e., viewer state remains if they have clicked on the recommendation) else x,, = 0 with probability Lx,z", (indicating that the viewer may leave the platform with probability Cr The expected measurement is the expected click rate when the recommender pulls arm k, i.c.,
E y Pkaet
Ta # 0) if [ =1
In Saig & Rosenfeld (2023), the viewer internal state di is set of tuples consisting of their previous interaction with the platform. That is, =x, {(b,, V", 1 < t} where by is time when j-th event happens, Vi, is the j-th recommendation viewer obtained and y is the viewer's reported rating. Depending on the subject of interest, V,, is defined differently as function of ah For example, y can be - b which is viewer's "instantaneous [response] rate."
In Laforgue et al. 20222. ; the recommendation yf,, defined the same as that of Levine et (011)). The internal state di a K-dimensional vector where xi = 64 - [k], -( where 0g transition function that tracks switches. The expected measurement when [A] 1 is B(y,,, m& m*( (k)) where mg is arm-wise reward function. In Poussoul et al. (002)), m. is assumed to be monotonic.
In Khosravi
2023), the recommendation yf,, is defined the same that of Levine
2017). The
viewer internal state is real-valued, i.c., x², IR. If yf 1, x., x/t- A(be -N,,- where by is an arm-wise constant and À is a universal constant. If vf, 1, then Elviel = rx I/,, where T E R is an arm-wise constant.
C Additional details for Content Creators in Section 3.2
The models for creator behavior can generally be broken down into two categories: models which focus on a single time step and treat creators as myopic, and models which consider non-myopic interactions across multiple time steps. In all of these models, the viewers are static.
Myopic models. We describe the myopic models (Ghosh & McAfee 2011
Ben-Porat & Tennenholtz 2018
et al.; 2020 Jagadeesan al.) 2022 Hron et al. 2022 Yao al. 2023a Dilat. & Rosenfeld
2023 Immorlica et al. 2024) where the creators (and often the recommender system) do not factor in
multi-time-stop interactions.
Before diving into the papers, we introduce some formalisms. Let. A be the action space of the creators which captures internal decisions about what content to create. Suppose that there are p 2 2 creators competing for recommendations. Each creator E [p] has internal state rf. = ag, specified by the creator's internal action a5,, E A. The output Vf af captures the content uploaded by creator j, the set 3/f = (15 Jelr) captures the content landscape, the output yf captures recommendations, and the output vf captures viewer behaviors such as clicks.
je(n)
23

---

We assume that at each time step t, the creator receives as input the recommendations vi and the prior content landscape vi-1- The state update Ie captures the creator's best-response to their utility function Uy "z vf') which depends on the action a of a creator j, the actions -= of other creators j' * in defined by Published in Transactions on Machine Learning Research (06/2025)
the prior content landscape 1f- and the recommendations yf- In particular, the state update is the besp-response:
G = (a vi-1), we) = AUg(a 1,)--111
The utility function Uj (and thus the state update f*) implicitly depends the following two functions. The first function is creator reward function h(j, vi, a)) which captures the reward that creator j receives from the measurements yi and yi along with production costs of their action a. The second function is the recommender function g" ("(x,, [ú, 1 y-- 8t), which maps the recommender state z and inputs (i.e., the user index i, content landscape 16-1, and viewer behaviors vE-1) to recommendationn yf- In the myopic models these papers, the recommender function 9d (af, [i. 1. .l. 04) only depends on the inputs i and yf as well the noise ve. We thus introduce the simplified function (g')r (i; vi- vz) to denote recommendations. (We formalize how the utility function U, depends h and (g')" after describing each paper.)
We specify cach paper in terms of how it defines the action space A, the creator reward function h, and the recommender function (g')":
Ghosh & McAfee 2011 take A = [0, 1] (capluring quality). The recommender function (g')² awards prizes to creators based on their quality rank. The reward h captures prize for creator j according to the recommendation ranks given by y" minus a 1-time cost of production c(a). The model additionally incorporates participation decisions.
Ben-Porat & Tennenholtz (2018) take A to be a finite set. The recommender function is randomized function (g') mapping each viewer to a content in y or no content (denoted by 9). The reward h captures the number of recommendations in y" assigned to creator j.
Ben-Porat et al. (2020) take A to be finite set capturing content. topics. When a given creator j writes topic a A, their article is predetermined quality qa.z- Each viewer seeks particular topic a A of content. The recommender function (g')" assigns a viewer secking a topic a the content with highest quality of that lopic: that is, jggmaxjejp Iaa af]- The reward captures the number of recommendations in y" assigned to creator weighted by topic-specific weights.
Jagadeesan et al. (0022 take A A R2 to be D-dimensional content embeddings in the nonnegative orthant. Each viewer has fixed preference vector u, E R. The recommender function (g')" assigns the viewer the content. created by creator that maximizes the inner product af)= that is, argmax (ui, af)- The reward h is the number of recommendations won minus the 1-time cost of production specified by c(a) - ||o||".
Hron et al. (2022) take A to be D-dimensional content embeddings in the unit sphere {>/||=[|a E RP). Each viewer has fixed preference vector u, RD. The recommender function (g')" assigns viewer i the content created by creator with probability proportional to ellu.))) The reward h is the mumber of recommendations won.
Yao et al. 2023a take A R be an abstract set. There is set of viewers X S RDD The recommender system computes scores for each content and viewer pair and assigns the top K recommendations to be the top K scores. The reward h is the sum of the scores of the recommendations won.
Yao et al. 2023b) take A PR to be an abstract set. There is a set of viewers X C RR, The recommender system computes scores for each content and viewer pair and determines recommendations (g')" based on arbitrary function of these scores and the viewer characteristies. The reward h captures the reward received by the creator (specified by general reward function of the scores and the viewer) minus the production cost c(a).
Immorlica et al. (2024) take A RR3 be 2-dimensional content embeddings capturing a clickbait dimension and quality dimension. Each viewer i has fixed type 8; >> 0 representing their tolerance for
24

---

Published in Transactions on Machine Learning Research (06/2025)
clickbait and only engages with content if they derive nonnegative utility (viewer utility increases with quality and decreases with clickbait). The recommender function g" optimizes an engagement metric (which is misaligned with viewer welfare) and assigns viewer i the content that maximizes viewer i's engagement. The reward is the mumber of recommendations won (that viewers actually engage with) minus a 1-time cost of production.
We specify the form of the creator utility function U,. The creator utility function Uj anticipates the impact of the creator's actions on recommendations in the next time step, which make it slightly messy to formalize in the dynamical system framework, but which we can nonetheless formalize as follows. At time step t, creators j first assesses how their actions a and the actions of other creators a -=jز- would affect the content. landscape
(a) == [oi -.... -at.aa--.......-------........
.
Then, the creator assesses the impact on the downstream recommendations
(a) = [(o' (44)), (00
and observed viewer behaviors
Th [a,+1(a)= = 9 (x (()) v++)).
and computes their utility:
a, = h(j (a), (.)))))))))))))
Recall that the papers described above study the equilibrium in the game between creators, which correspond to fized points of these dynamics. More specifically, the fixed points of the dynamics described above correspond to the pure Nash equilibria. Several papers also consider mixed Nash equilibria, but expressing these equilibria fixed points requires introducing randomness into the dynamics and carefully accounting for who observes the rcalizations of the randomness.
We note that some myopic models do not fit into structure that we described above. For example, Filat & Rosenfeld (2023) models the recommender system as choosing (or learning) user embeddings. The user embeddings affect recommendations through a fixed ranking function and a fixed bipartite relevance graph. Each creator chooses the content that maximizes the average (linear) relevance score for embeddings of users connected to them in the relevance graph.
Non-myopic models. We next turn to the non-myopic models (e.g. Ghosh & Hummel 2013 Hu et al. 2023 Bucning et al. 2023). where creators and the recommender system factor in multi-time-step interactions. These models of content creator behavior are generally more complicated to formulate. The recommender function g" is modelled as a multi-bandit algorithm. Creators account for their (potentially discounted) cunulative reward when selecting actions.
Ghosh & Hummel 2013) consider setup where each creator j enters the platform at potentially different time step. The action set is A= [0, 1] and captures content quality. The recommender system is a stochastic multi-armed bandit algorithm. The reward h is the number of recommendations to be won at the current round and in the future minus the cost of production c(a). The model additionally incorporates participation decisions.
Hu et al. 2023) consider a setup where all creators arrive al every time step and can create new content at each time step. The action set A E (R.2))T The recommender system an adversarial multi-armed bandit algorithm. The reward h is the cumulative discounted sum of the number of recommendations to be won minus the cost of production at each time step. Production costs at each time step t are specified by c(a) = ||a]| This model makes the simplifying assumption that each creator chooses the content that they will produce at every time step at the beginning of the game.
Buening et al. 2023) consider a setup where creators all arrive at every time and each choose the feedback rate a E A = 1) of their content at the beginning of the game. The recommender system is a multi- armed bandit algorithm that accounts for probabílistic feedback. The reward is the total mumber of recommendations won.
25


Reviews:
[{"review_id": "yQ91ppUBHU", "reviewer": "Reviewer_dYcT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\nThe paper is well-written, clearly structured, and makes a clear claim for its contribution. The linguistic style is accessible and easy to follow, the mathematical formulations are to the point and can be grasped quickly. The paper introduced to me the idea of formal interaction models, which I found interesting. The paper uses several examples to guide understanding and identifies several implications and avenues for future work. All in all a thorough and pleasant paper.\n\n## Weaknesses\nWhile the paper's claims regarding how formal interaction models can be used and what they can achieve are not implausible, I was wondering multiple times how these aims should be fulfilled precisely. As an example, the introduction states that formal interaction models can, in extension, be useful to build reliable AI systems with positive social impact. Examples of applications are \"building, evaluating, deploying, and auditing AI systems\". However, after reading the paper, I do not necessarily have a clearer idea of how formal interaction models can be used to build reliable AI systems nor can I picture them in, e.g., AI auditing. The paper demonstrates how formal interaction models can be used to formalize user boredom with respect to a recommender system, but how does this translate into societal impact? Similar questions arose regarding the \"use cases\" of formal interaction models (specifying, monitoring, anticipating, controlling). While the cited literature reflects all these aspects, it did not become clear to me how the formalization of interaction supports these use cases specifically. Here, I feel that a more thorough running example would improve the tangibility of concepts and claims.\n\nRelated to this, I did not get a clear picture of the level of abstraction in which formal interaction models are suggested to be applied. (Although here as well I was wondering how the exact terms came to be defined, such as the state update 0.5 * x + y.) I can imagine a recommender system employing the user boredom model as an input to its content ranking. In contrast, I find it difficult to picture how and which formal interaction model can represent polarization and addiction (Section 3.2). Addiction, for example, seems to include many variables that would need to be considered in this model, such as psychological conditions (addictive tendencies, social environment), interaction features (infinite scrolling), and type of content. Do the authors suggest that these aspects are captured with noise variable w (this variable seems to be underspecified)? Or are formal interaction models not supposed to represent concepts like addiction in detail? In short, the paper does not fully provide enough argumentative ground or examples to grasp how formal interaction models impact society-level issues. Here, I invite the authors' clarification.\n\nParts of these questions might also be answered through an extension of Section 2.3, which describes design choices for formal interaction models. On the one hand, I wondered how these design axes were selected, and on the other, I would appreciate more concrete examples of how these models can have different styles and granularities. Another suggestion to address these questions would be the extension of the discussion, which currently is quite short.\n\nLastly, I am missing a Section that describes how papers for the literature survey were selected and analyzed. While I understand that the paper does not claim that it conducted a systematic literature review, some information on how the concepts presented were synthesized would be welcome.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper argues for the development of formal interaction models -- a formalization of interaction between human users and an AI system. These models are based on the framework of coupled dynamical systems, which the authors suggest to serve as a unifying system to represent all sorts of interaction models. The authors focus on recommender systems to illustrate the functioning and benefits of formal interaction models, building examples that include a recommendation system, a viewer, and a creator, which reciprocally influence each other. Users' time spent is understood as the input to the recommendation system, while suggested content is understood as it's output, each serving to measure and update states in user and system. The authors posit that formal interaction models can be used for four main points: specifying user interactions, monitoring interactions, anticipating societal impacts, and controlling societal impacts. The authors add a review of literature investigating viewer preferences and creator content and cast both concepts into formal interaction models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Note that these are suggestions:\n- Introduction of a running example to illustrate the claimed use cases of formal interaction models\n- Inclusion of a methods section or similar to outline survey approach\n- Discussion of how complex user preferences and adverse system effects are represented in formal interaction models, see for example [1,2,3]\n- Discussion to address relation between formal interaction models and societal impact\n- Inclusion of a conclusion\n- More details or examples of different models' styles and granularities\n- Clarification on how the state update and measurement equation in Example 1 were defined\n- Grammar and typography check, I found several word repetitions and misspellings\n- Workover of the second-to-last paragraph on page 9, here the text is hard to parse due to the references\n\n[1] Shulner-Tal, Avital, Tsvi Kuflik, and Doron Kliger. \"Fairness, explainability and in-between: understanding the impact of different explanation methods on non-expert users’ perceptions of fairness toward an algorithmic system.\" Ethics and Information Technology 24.1 (2022): 2.\n[2] McInerney, James, et al. \"Explore, exploit, and explain: personalizing explainable recommendations with bandits.\" Proceedings of the 12th ACM conference on recommender systems. 2018.\n[3] Schmude, Timothée, et al. \"Information that matters: Exploring information needs of people affected by algorithmic decisions.\" International Journal of Human-Computer Studies 193 (2025): 103380.", "broader_impact_concerns": "There was only one point in the manuscript where I briefly noted broader impact concerns. In Section 2.3, measurability is discussed as a design axes of formal interaction models. The last sentence in this paragraph already hints at the possibility that some concepts are unobservable. Dysfunctional AI systems are often based on the assumption that aspects such as psychological well-being or employability are easy to quantify and predict. I would like a brief description of how formal interaction models would deal with these unobservable constructs and what the (maybe ethical) implications for their designs are. To this end, I recommend: Jacobs, Abigail Z., and Hanna Wallach. \"Measurement and fairness.\" Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 2021.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kJjWiZr923", "reviewer": "Reviewer_2QUS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This work discusses a great number of use cases for this problem. I think incorporating the current methodologies (off-policy learning, contrastive methods, etc) for solving this problem, and discussing their limitations, and probable future steps, will further improve the contributions in this work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This survey paper pushes the AI community for the need to address various distribution shifts (user preferences, content distributions, etc) by formal mathematical modeling. \nP.S: This is a short review as I am in an emergency situation myself.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "-", "broader_impact_concerns": "-", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yUkA5HlppI", "reviewer": "Reviewer_u8TQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- Structured and comprehensive approach: The paper delineates multiple practical use-cases for formal interaction models (implementation, empirical monitoring, societal impact anticipation, and control). The outlined framework is precise and helps bridge theory and practical application.\n\n- Insightful case study analysis: The authors analyze the existing literature on recommender systems, highlighting gaps and future research opportunities. By applying the framework, they illustrate the potential utility of formal interaction models.\n\nWeaknesses:\n\n- Limited empirical validation: The theoretical nature of the paper lacks empirical demonstrations to showcase practical effectiveness.\n\n- Insufficient integration with existing frameworks: The paper doesn't clearly position itself relative to established simulation and modeling methods (e.g., reinforcement learning environments).\n\n- Abstract practical guidelines: The advice on navigating trade-offs regarding measurability and complexity lacks concrete examples or clear guidelines.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces the concept of formal interaction models, mathematically specified as coupled dynamical systems, to explicitly capture how AI systems and users reciprocally shape each other's states and behavior. The authors articulate four key use-cases for these models: (1) specifying interactions for implementation, (2) monitoring real-world interactions empirically, (3) anticipating societal impacts through counterfactual analyses, and (4) controlling societal impacts via algorithmic or policy interventions. They provide a thorough and structured overview of the design considerations for these models, inluding style, granularity, mathematical complexity, and measurability. A case study of content recommendation systems illustrates and critically assesses existing literature and underscores both the utility and limitations of formal interaction models.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### Empirical Validation (Critical):\n\nInclude empirical validation or illustrative simulation results, using real-world datasets or established simulation environments (e.g., RecSim, recogym). Demonstrating how formal interaction models can empirically capture or mitigate unintended societal impacts would substantiate theoretical claims.\n\n### Enhanced Comparison to Existing Approaches (Important):\n\nClearly position the contribution relative to established frameworks such as model-free and model-based RL, causal inference, or economic models. Consider providing a brief comparative table or figure to clarify the added value and differentiation of formal interaction models from these established frameworks.\n\n### Detailed Guidelines on Practical Implementation (Moderate):\n\nProvide explicit examples or scenarios illustrating how to practically navigate the trade-offs along the axes of granularity, measurability, and complexity.\n\nConsider adding concrete examples or short case illustrations in a dedicated appendix to clarify model design decisions clearly.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FAuYcar4CM", "reviewer": "Reviewer_q8D4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "S1: I like the premise of the paper. We often treat AI systems as standalone things. But in the real world, they interact with humans (and other AI systems) and the dynamics behind those interactions might change over time.\n\nS2: The writing and its flow are great. It's almost a pleasure to read. \n\nS3: I think the engagement with prior work seems pretty thorough and does not seem perfunctory. I'm not an expert unrelated literature so I can't comment on whether or not things are missing. But it looks like the authors did their homework.\n\nW1: The paper is lacking an illustrative & pretty figure 1 to convert the main couple ideas of the paper. Consider adding one if appropriate. \n\nW2: the central example of content recommendation seems to highlight another challenge with modeling humans, and a shortcoming of past works' modeling of humans. Humans have different orders of preferences. For example, my lizard brain might want me to keep scrolling through cat videos though another part of my brain is telling me to watch educational content. It seems a little bit odd to address only one of these problems at a time. For a problem like this, we need to consider the fact that human preferences can change, but we also need to be wary of the complex, multifaceted, and contextual nature of human preferences themselves. Overall, I don't think this is really a problem with the contribution of the paper, but it reflects a slight uneasiness I have with the way the paper discusses things and motivates using examples.\n\nW3: this isn't really much of a weakness. I think this paper is a really good survey paper and will be valuable. But I usually think that survey papers are a bit better if they do something on top of the survey, such as some quantitative analysis about what's going on in the field, some experiments, etc. Again, this isn't a real weakness for me. But if the authors can think of any useful things to add, that might be interesting.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In computer science and machine learning, it is very common to model and study systems in a way that doesn't take the complexities of their interactions with humans into account. In the field of human computer interaction, it is very common to try taking these interactions into account, but most work fails to address the possibility that, through interaction, the system and user can change. For example, after interacting with a recommendation algorithm, my content preferences may change overtime. This is a survey paper (which is lowkey a little bit of a position paper) that is dedicated to the use of human/computer models that allow for change over time in HCI. Most of it is dedicated to surveying and discussing these types of models, including how/why they can be used as well as a case study into the broad challenge of studying content recommendation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Above, I asked the authors to consider a few things. Please consider them. However, overall, I think it is obvious that this paper is really thorough and well-written. I think it does what it has set out to do very well. If I knew the literature behind this work better, I might have some specific advice for how to improve discussion of specific work. But I do not. To me, this paper seems ready as is with potentially minimal changes.\n\nOverall, I think this paper has the potential to be valuable. And most of all, I just think it has been well executed. I will recommend acceptance unless other reviewers have major concerns.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper received four reviews containing several suggestions for improvements (mainly regarding clarifications, e.g., adding/expanding on running examples, and expanding on the relation to other modeling methods, e.g., reinforcement learning environments). The authors have updated their paper in line with these suggestions and all four reviewers recommended the acceptance of the paper. Thus, as the claims of the paper are substantiated and the paper is relevant to the TMLR audience, I am recommending the acceptance of the paper.
Decision: Accept (poster)