Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper suggests to view the strategic behaviour of _users_ of agent representatives, assuming those representatives are playing a repeated normal form game using regret-minimisation. As opposed to many papers that would present a limited set of specific examples, authors present analysis of game _classes_ where users are an incentive to mis-represent their actual preferences to manipulate the interaction between the automated representatives. 
strengths_and_weaknesses: Strength: Analysis of manipulation of adaptive learning _systems_. Applicable (potentially) to multi-agent learning scenarios, especially hierarchical learning approaches. Game _class_ analysis.

Weaknesses: Assumption of an equilibria seeking behaviour. Needs at least a mention of off-equilibria behaviour possibility (see works by 
questions: There are authors who study out-of-equilibrium behaviour and "stable cycles". Are there any indications of what happens if the automated agents are locked into cycle/dynamic-equilibrium patterns, rather than strongly converge to a (pure) equilibrium in beliefs/traces? 

Along this same line of thinking. Since the model already admits separation of representative and user utility, what happens if the meta-game utility is even further removed. E.g., trading on "futures", deriving utility from the representative agent beliefs, rather than decisions that those beliefs produce in the underlying game at that moment?

limitations: none beyond the questions above
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors consider a setting where human players play a game, but automated agents execute their decisions. This scenario arises in, e.g., online advertising, where an automated bidding agent makes real-time decisions intending to maximize the utility of the player it represents. The authors distinguish between the meta-game between human players and the game instantiation the agents play. The instantiation is induced by players specifying parameters like their utility for outcomes (valuations, costs, etc.). The authors claim that automated agents are regret-minimizing algorithms in many settings.

The authors identify a loophole in this modeling: Aiming to maximize their utility, the players might be incentivized to manipulate their agents. Namely, there are cases in which passing the wrong parameters to agents can actually lead to better player utility. 

The authors analyze three classes of games and show when/how manipulation can help players. They provide elaborate examples along with the formal characterization of beneficial manipulations. 
strengths_and_weaknesses: Strengths:

1. The topic is hot and relevant. Real-world applications that employ automated agents (like online advertising) are positioned in multi-billion industries. Even small advancements can be highly profitable, so reasoning about potential harms is highly impactful.

2. The claims are non-intuitive. Regret-minimizing algorithms are used widely, and the authors prove they can become problematic in multi-agent settings. Furthermore, the authors provide deep insights like the connection between regret-minimizing to Stackelberg games and cooperation in Theorem 1.

3. The paper is full of examples that assist in understanding the different classes of games, the deviations, and the theoretical results.

Weaknesses:
1. My main concern about this paper is the so-called companion paper (first presented in line 80). The authors, it seems, already initiated the study of "incentives of users in automated learning agents in online strategic systems" with that companion paper (assuming it was published). Hence, line 67 is misleading: only one of those papers could initiate this study. Further, I'm not familiar with treating previous literature as a "companion paper" --- how should I assess the contribution of this paper without knowledge about the other? I believe the authors should have addressed their previous "companion" paper as related work in this paper. I understand that addressing the companion paper risks anonymity, and yet the current situation where there's another paper on this topic, but it is not surveyed here is also unfortunate. 

2. If I understand correctly, the incentive misalignment follows from the automated agents being regret minimizers. Therefore, I was surprised to see that the authors don't formally define the notion of regret. Further, they discuss no-regret dynamics or other essential notions such as CCE without defining them. I believe that defining these technical notions is crucial for understanding the paper's contribution (and appreciating the paper's non-intuitive claims about the incentive mismatch). 

3. I did not consider the following comment in my score, but I believe the authors could significantly improve the write-up. Here are several examples:
* The order of citations is incorrect (e.g., [75, 43, 5] in line 127, instead of [5, 43, 75])
* Many sentences are unclear. For instance, line 143 says, "this simple analysis also shows that the other player has..." Who is the other player? The word "this" at the beginning of the sentence is a dangling modifier. Many other sentences were hard to follow.
* Inconsistency of technical details, for instance, "sub-family … [of] games" vs. "subclass of… games" in lines 262 and 264. Such inconsistencies make reading harder.
* Wordiness, for example, line 222: "Analyzing the equilibrium of the meta-game, one obtains that the equilibrium of the meta-game." Why should we repeat "equilibrium of the meta-game" twice? I believe that the authors could be more articulate in this sentence. 


questions: Overall, I'm positive about this paper but want to understand the author's stance about the weaknesses I raised. Please address my 1 and 2 concerns in the "weaknesses" part.
limitations: Null
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces the study of how users interact with game-solving agents - in particular, it explores the question of what happens if we allow a user to specify, or indeed *mis*specify, her own utility function to a learning agent that will then attempt to learn and play an equilibrium on behalf of the user. The paper then establishes, somewhat counterintuitively, several cases in which the user has incentive to misspecify her utility, despite the fact that the learning agent's objective is perfectly aligned with the user's, and carefully analyzes the Nash equilibria of the resulting "meta-game" for the users in these various cases.
strengths_and_weaknesses: This paper is very intellectually stimulating and well written, and I think it brings to light an important and quite counterintuitive concept. For that, I think it is worthy of acceptance. 

If I can point out any weakness, it is perhaps the specificity of the theoretical results: they are limited to special cases of dominance-solvable, Cournot, or 2x2 games. I do not think that this weakness is significant though--it simply leaves room for better future work!

I also think that one of the points raised only in the appendix--namely, that regret minimization alone is never guaranteed to converge to any particular CCE in a game with multiple CCEs--is interesting in its own right, and perhaps deserves a mention in the main body.
questions: 1. Is there anything special about regret-minimizing agents in the context of this paper? It seems as though all the results in the paper hold if the learning agents are replaced with arbitrary black-box CCE (or Nash) solvers. If this is the case, I think it'd be better to frame the result in this light, as it would be more general.
2. To what extent is it possible to "mechanism design" around this problem? For example, can one program the learning agents in such a way that the users are incentivized to tell the truth? At what cost does this come? For example, such learning agents can't always converge to CCE, lest they be subject to the incentive problems raised by this paper, and therefore they cannot all be regret minimizers. Can we say anything positive?
3. Thm 4 reminds me of the revenue equivalence principle in mechanism design. Is there any connection here?
limitations: Yes.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper investigates equilibria of a class of competitive games where players do not take actions directly, but instead pick a configuration for an (automated) learning agent which then engages in a multi-round competition with agents configured by the other players. The configuration is assumed to only affect the “declared utility” a player derives from each round’s outcome. The agents are assumed to employ a no-regret learning algorithm which maximises the *declared* utility. Under the authors’ assumptions, the studied games always have a coarse correlated equilibrium (CCE), implying the statistics of the repeated game _between the agents_ converge to the CCE in the infinite horizon limit. The authors then study the “meta-game” *between the players*, and their incentives to declare utility different from their underlying true one (which is assumed private). 

The authors study three classes of games played by the agents: (i) dominance-solvable; (ii) (linear) Cournot competition; and (iii) opposing-interests games. In terms of “stability”, they find that players often have an incentive to declare utility different from their true one (“deceiving their agent”). Whether players achieve greater/lower utility in the CCE then depends on which of the above three agent games is played. The authors provide rigorous characterization of both (and say that deeper investigation of repeated auction is in a complementary paper, presumably also submitted to NeurIPS).
strengths_and_weaknesses: **Strengths**
+ With an increasing number of algorithms deployed in the real-world, scenarios where agents interact with each other are likely to increase in frequency. Understanding the resulting dynamics is therefore useful.

+ Breadth: The authors investigate a relatively large number of theoretical settings (dominance-solvable, Cournot, and opposing-interests games).

+ The authors fruitfully connect a variety of concepts from machine learning and economics.


**Weaknesses**
- AFAICT the authors do not provide a single reference to where a setup like theirs appears in the *real world*. The authors mention, e.g., auctions, but I am not aware of any instances where the *auctioneer* provides a _fixed_ (adversarial bandit) agent, with which the customers interact solely by declaring outcome utilities. IIUC, the assumption of an external fixed agent and utility declaration are crucial for relevance of these results, as more direct control over the agent would result in a standard repeated game (unless other constraints are introduced ofc). (Repeated games have ofc been studies extensively.)

- Relatedly, the paper explains technical definitions of the dominance-solvable/Cournot/opposing-interests games, but does not provide justification for why these particular games should be of interest (based on real-world applications).


questions: 1. Can you please provide real-world examples for where the setup you study exists? (See the “Strengths and weaknesses” review section.) If you can do so convincingly, I will change my recommendation to accept.

2. Can you please clarify why you are using the “single-shot” Nash equilibria as the baseline as opposed to some type of equilibria of the repeated game?

3. Is there ever an incentive to hire more than one SW agent? (In other words, how is the assumption of one agent per player justified?)
limitations: The crucial limitation of this work is in its motivation/applicability to realistic scenarios, as already discussed in other parts of the review. If there are none, it is hard to see merits; if there are, it is a good paper.

I did not look at the appendix which includes proofs.
review_title: null

Meta-review: metareview: Most reviews are positive and think that the paper solves an interesting and non-trivial problem. One reviewer points out some concerns on the motivating example, and it seems to be addressed in the author response. I have a different concern that in real world, the set of eligible bidders in each auction differs a lot, and maybe the authors can add some discussion on how this affects the result.
confidence: Less certain
Decision: Accept