Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: This paper discusses how Coalitional Bargaining Games (CBGs) with non-repeating proposals are non-Markovian in nature and proposes a way to make these CBGs Markovian. 
strengths_and_weaknesses: S: Well written and good first step.
W: The main text has little technical content
suggested_changes: 1) It would be good to reduce the introduction somewhat and include a few more technical details in the main paper. Including Figures 1 and 2  in the main paper would also allow readers from other domains to understand the proposed concepts better. 
2) Typo in Section 1: "The the state dynamics are determined by the agent’s preferences and the rewards are assigned once a proposed coalition is accepted."
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: The paper presents a method for converting a non-Markovian coalition game into a Markovian game through the use of filtrations.  This conversion allows existing RL methods to be applied to coalition-based games.  Two short proofs are presented.
strengths_and_weaknesses: The paper is relatively clearly written, and is really quite enjoyable to read.  The “findings” are simple, but this is certainly not a bad thing.  

I believe the paper is correct, since the proposed method is relatively simple and no empirical methods are presented.  I have outstanding questions about why invoking the relatively complex and off-putting sigma-algebra is strictly necessary.  I feel like one could simply consider an extended state space (with dimension $N\mathbb{C}1 + N\mathbb{C}2 + … N\mathbb{C}N$), and that is somewhat easier to understand.  When I trial a coalition, I deterministically set that extended state to zero.  You present it in terms of the sigma-algebra, and I'm not yet convinced (although I am not an expert on such things) that this is more natural than extending the state space.  Furthermore, how is the sigma-algebra implemented in practice?  If I wanted to implement this in Python, what does this correspond to? 

There is an element of “deus ex machina” about the presentation as.  “We cannot do something because it is non-markovian, but, there is a special operator that makes it markovian, and now the games are markovian”.  This is obviously **very** reductionist of me, but it makes me feel like I am misunderstanding the full potential and generality of this work.  I invite the authors to convince me, or, in lieu of a rebuttal, include more discussion as to the practicalities of the filtration operator, and any technical information required to apply it in this domain.

To summarize: The idea of converting a non-Markovian game to a Markovian game, such that existing tools can be applied, is super super appealing, and I think would have substantial impact.  I am just unsure on whether or not the implications of using a filtration operator have been fully fleshed out, and hence the original “research” question actually still applies.  I hope the authors respond to this, because otherwise, it is a very enjoyable TinyPaper. 

suggested_changes: I have some minor suggestions to change the manuscript, which I outline below.  

My main request is that the authors disambiguate/discuss how the sigma-algebra relates to the idea of a Markov game on an extended state space, which I referenced above.  

Minor:
- Section 2: you provide two inequalities, eg. $| c_1 \cap c_0 | < | c_0 |$.  I don’t understand how either constraint could ever be violated (excluding an equality), and hence I don't understand what it is trying to specify. I could believe that c1 could “add” a node to c0, in which case the cardinalities would be equal, and this would be violated, but i don’t see why that should be excluded as a case.

Typos:
- Pg 1: “...one agent at random propose[s] a coalition…”
- Footnote 1: The “1” typically comes after the period.  Also, the footnote itself should start with a capital letter (“[T]o simplify…”)
- Pg 1: Missing space between “proposal.” and “The”
- Pg 1: Duplicated “The”.
- Throughout: citation style is incorrect.  E.g. the “Okada (1996)” should be formatted as “(Okada, 1996)”, as the work “Okada” should ot feature in the sentence.  For instance, later, “Bachrach et al. (2020)” is correct, because the word “Bachrach” should be in the sentence.
- Pg 1: the sentence containing “... but introduces learning agents _learn_ avoiding …” is poorly worded, to the point where i don’t understand what it is trying to say.
- Throughout: Only proper nouns should be capitalized, eg. “Markov [p]erfect Nash [e]quilibium”.
review_title: null

Decision: Invite to present